AIResearchGenerative ModelsOptimal Transport

Improving Function Space Flow Matching with Kernel Optimal Transport

kFFM replaces the random noise–data pairing of Functional Flow Matching with entropic optimal transport under a kernel-induced cost, the coupling behind the Hilbert Sinkhorn Divergence: a signature kernel for paths, a Sobolev or Euclidean RBF for fields. Same network, same loss, 5–7% more training time, and better samples on eight time-series and PDE benchmarks.

Paper Code

Three panels: bump-shaped data curves each drawn with a dashed noise curve as its partner, unrelated under random pairing and rising where the bump is under kFFM pairing, and two histograms of residual size, the kFFM one shifted to smaller values
Pairing functions (synthetic). Data are bumps at random locations and heights (solid), and the noise is a Gaussian process (dashed, same color as its partner). (a) Under random pairing a bump’s partner has nothing to do with it. (b) Under the kFFM coupling, partners tend to rise where the bump is. (c) Over 200 batches of 64 pairs, the residuals the network must learn are about 21% smaller on average.

This post introduces our NeurIPS 2026 paper, “Improving Function Space Flow Matching with Kernel Optimal Transport,” written with Thomas Markovich of Block, Barbora Barancikova of Imperial College London and Yizhou Sun of UCLA. Many scientific datasets are made of functions, such as temperature curves, gene-activity time series, volatility paths and snapshots of a fluid. Functional Flow Matching (FFM) learns to generate such data by gradually turning random functions, the noise, into realistic ones, but during training it pairs each noise sample with a data sample at random. Our method, kernel Functional Flow Matching (kFFM), pairs each noise sample with a data sample that already resembles it, by solving a small optimal-transport problem in every training batch under a notion of distance designed for functions: a signature kernel for paths, and an RBF kernel on a derivative-aware Sobolev norm for fields. Nothing else changes, neither the neural network nor the training loss, and training takes 5–7% longer at the largest resolution we train. We prove that this kernel pairing is well-posed and bounded on general function spaces, split its gap to standard optimal transport into interpretable terms, and show that it is stable as the grid is refined. On eight time-series and PDE benchmarks, kFFM has the lowest MMD of all compared methods on every dataset and beats FFM with seed-paired significance on all eight (on three of them while also using a different noise distribution), and the gains hold under non-kernel metrics and on a turbulent Navier–Stokes benchmark.

Why generate functions?

Some data are best thought of as functions rather than as lists of numbers. A weather station’s average temperature over the year is a curve, and so is the activity of a gene over time, or a country’s population from 1950 to 2018; the variance of a stock in a stochastic volatility model traces a rough, jagged path; and the vorticity of a two-dimensional fluid is a field, one number at every point of a square.

Three panels: one temperature curve over a year sampled at 365, 52 and 12 points; several rough, mean-reverting paths; and a red-and-blue random field on a square
Three kinds of function-valued data (synthetic examples). (a) The same smooth curve recorded monthly, weekly and daily: the grid is a choice of the person recording it, while the underlying object is the curve. (b) Rough, mean-reverting paths of the kind produced by stochastic volatility models. (c) A random field on a 2D domain, like the vorticity of a fluid.

A generative model for such data learns their distribution and can then produce new, realistic samples, such as plausible temperature profiles, volatility scenarios or fluid states, and two properties make this setting different from generating images or vectors. The first is that the grid is not the object: the left panel above shows one curve at three resolutions, and a good model should learn the curve rather than the grid, so that it can be trained at one resolution and sampled at another. The second is that geometry matters, since two functions can be close or far apart in ways a flat list of grid values does not capture, one being a slightly shifted copy of the other, or the same shape with fast wiggles on top, or a path with the same overall trend but a very different roughness.

Our paper improves one step of a strong existing method for this problem, and it draws on more mathematics than most machine-learning papers: function spaces, probability measures on them, optimal transport, kernels and path signatures. The first half of this post builds these up one at a time, each with pictures, and the second half uses them to explain kFFM, its theory and its results. Readers who already know a topic can skip its section, and the method itself is summarized in the section “kFFM on one page.”

Functions as points in a space

To do geometry and probability with functions, we treat each function as a single point in a space. Functions can be added and scaled pointwise, so they form a vector space, but an infinite-dimensional one, and the workhorse is L2(Ω)L^2(\Omega), the square-integrable functions on a domain Ω⊂Rd\Omega \subset \mathbb{R}^d, with inner product and norm

⟨f,g⟩=∫Ωf(x) g(x) dx,∥f∥=⟨f,f⟩. \begin{gathered} \langle f, g \rangle = \int_\Omega f(x)\, g(x)\, dx, \\ \|f\| = \sqrt{\langle f, f \rangle}. \end{gathered}

A Hilbert space is an inner-product space that is complete, meaning that sequences whose terms get arbitrarily close to each other converge to an element of the space. Like Rd\mathbb{R}^d, a Hilbert space has orthonormal bases: on a periodic interval the Fourier modes en(x)=e2πinxe_n(x) = e^{2\pi i n x} are one, and every function is the sum of its Fourier series,

f=∑nf^n en,f^n=⟨f,en⟩,∥f∥2=∑n∣f^n∣2, \begin{gathered} f = \sum_{n} \hat f_n\, e_n, \qquad \hat f_n = \langle f, e_n \rangle, \\ \|f\|^2 = \sum_n |\hat f_n|^2 , \end{gathered}

so that a function is an infinite sequence of coefficients, an “infinite-dimensional vector.”

A computer stores finitely many numbers, grid values or, equivalently, the first few Fourier coefficients, so discretization is projection: keeping the modes up to NN is the orthogonal projection PNf=∑∣n∣≤Nf^nenP_N f = \sum_{|n| \le N} \hat f_n e_n onto a finite-dimensional subspace VNV_N. Both the neural operator and the Sinkhorn solver in kFFM work with such truncated representations, so we will need to know how much is lost.

Sobolev spaces measure smoothness. Differentiating multiplies the nn-th Fourier coefficient by 2πin2\pi i n, so a function with kk square-integrable derivatives has coefficients that decay fast enough for

∥f∥Hk2=∑n(1+∣n∣2)k ∣f^n∣2 \|f\|_{H^k}^2 = \sum_n (1 + |n|^2)^k\, |\hat f_n|^2

to be finite; this norm is equivalent, up to constants, to ∫f2+∫(f′)2+⋯+∫(f(k))2\int f^2 + \int (f')^2 + \dots + \int (f^{(k)})^2. The functions for which it is finite form the Sobolev space Hk(Ω)H^k(\Omega), with H0=L2H^0 = L^2 and a larger kk meaning smoother functions, and solutions of PDEs naturally live in Sobolev spaces. Smoothness and decay go together, for a jump gives coefficients of size 1/n1/n, a kink 1/n21/n^2, and an analytic function decays exponentially.

This tells us how much discretization loses. For a function in HαH^\alpha and a weaker norm HkH^k with k<αk < \alpha, one line of algebra bounds the truncation error,

∥f−PNf∥Hk2=∑∣n∣>N(1+∣n∣2)k−α(1+∣n∣2)α∣f^n∣2≤(1+N2)−(α−k) ∥f∥Hα2, \begin{aligned} &\|f - P_N f\|_{H^k}^2 \\ &\quad = \sum_{|n| > N} (1+|n|^2)^{k-\alpha}(1+|n|^2)^{\alpha} |\hat f_n|^2 \\ &\quad \le (1+N^2)^{-(\alpha - k)}\, \|f\|_{H^\alpha}^2 , \end{aligned}

so that the smoother the function relative to the norm we measure in, the faster the error vanishes. This inequality is the engine of our discretization theorem, which we will meet later.

Three panels: a triangle wave and its projections onto 1, 3 and 9 Fourier modes; Fourier coefficients of a smooth function, a triangle wave and a square wave on log axes, decaying at different rates; and truncation error curves that fall faster for smoother functions
(a) A triangle wave and its projections onto the first 1, 3 and 9 Fourier modes. (b) Fourier coefficients of three functions: a jump (square wave, proportional to 1/n1/n), a kink (triangle wave, proportional to 1/n21/n^2) and an analytic function (exponential decay). (c) The L2L^2 truncation error ∥f−PNf∥\lVert f - P_N f \rVert shrinks faster the smoother the function.

In infinite dimensions a closed, bounded set need not be compact: the basis vectors e1,e2,…e_1, e_2, \dots all lie on the unit sphere, at distance 2\sqrt{2} from each other, so no subsequence converges. Smoothness restores compactness, since a ball of smoother functions, {f:∥f∥Hα≤R}\{f : \|f\|_{H^\alpha} \le R\}, is compact in the HkH^k norm when α>k\alpha > k, which is the Rellich–Kondrachov theorem: by the inequality above, every function in it lies within R(1+N2)−(α−k)/2R(1+N^2)^{-(\alpha-k)/2} of the finite-dimensional space VNV_N. The quantitative version of compactness is the covering number N(δ)\mathcal{N}(\delta), the number of balls of radius δ\delta needed to cover the set, and our error decomposition uses it.

Some natural spaces of functions have a norm but no inner product, such as the continuous functions with the maximum norm, or the paths of finite pp-variation that we will meet with signatures. A complete normed space is a Banach space, and the tools that rely on inner products, such as orthonormal bases, projections and the Gaussian measures of the next section, do not carry over to it directly, which is why it matters that our first theorem holds on Banach spaces.

Random functions and Gaussian measures

A generative model for functions is a probability distribution on a function space, and probability in infinite dimensions has one big surprise that shapes everything that follows: there is no uniform measure, and so there are no densities. In Rd\mathbb{R}^d a probability density p(x)p(x) is defined relative to volume, the Lebesgue measure, which is the same everywhere, but on an infinite-dimensional space no such measure exists. The reason fits in one sentence: the unit ball contains infinitely many disjoint balls of radius 1/41/4, centered at en/2e_n/2 for each basis vector ene_n, and a translation-invariant measure would have to give them all the same mass, so either everything has mass zero or the unit ball has infinite mass. A “density” then only makes sense relative to another measure: μ\mu has a density dμ/dνd\mu/d\nu with respect to ν\nu, its Radon–Nikodym derivative, exactly when μ\mu is absolutely continuous with respect to ν\nu, written μ≪ν\mu \ll \nu, meaning that every set of ν\nu-probability zero also has μ\mu-probability zero.

The best-understood distributions on a Hilbert space are Gaussian. A Gaussian measure N(m,C)\mathcal{N}(m, C) is a distribution under which every linear measurement ⟨f,h⟩\langle f, h \rangle is a one-dimensional Gaussian, with mean ⟨m,h⟩\langle m, h \rangle and variance ⟨Ch,h⟩\langle C h, h \rangle. The covariance CC is a positive, symmetric operator with eigenpairs Cen=λnenC e_n = \lambda_n e_n, and it must be trace class, ∑nλn<∞\sum_n \lambda_n < \infty, for a reason given by the Karhunen–Loève expansion, the infinite-dimensional version of “a Gaussian is a linear map applied to independent standard normals,”

f=m+∑nλn ξn en,ξn∼N(0,1) independent,E ∥f−m∥2=∑nλn. \begin{gathered} f = m + \sum_n \sqrt{\lambda_n}\, \xi_n\, e_n, \\ \xi_n \sim \mathcal{N}(0, 1) \text{ independent}, \\ \mathbb{E}\,\|f - m\|^2 = \sum_n \lambda_n . \end{gathered}

The sum of the eigenvalues is the expected squared size of a sample, so it must be finite for samples to be functions at all. A Gaussian process (GP) with covariance kernel k(x,y)k(x, y) is the Gaussian measure whose covariance operator is (Cf)(x)=∫k(x,y)f(y) dy(Cf)(x) = \int k(x, y) f(y)\, dy, and a smooth kernel gives fast-decaying eigenvalues and smooth samples.

White noise is the case C=IC = I, with λn=1\lambda_n = 1 for every nn. The sum diverges, so white noise is not a random element of L2L^2: on a fixed grid it looks harmless, but its variance is spread evenly over all modes, so it gets rougher as the grid is refined and never settles down to a function. FFM uses GP noise, while kFFM treats the noise distribution, called the base measure, as part of the configuration chosen for each dataset, a GP where the data are smooth and white noise on the grid where the data are rougher than a smooth GP prior, as in the Navier–Stokes family.

Four panels: smooth random curves sampled from a Gaussian process; jagged white noise on a grid; covariance eigenvalues decaying quickly for two Gaussian processes and staying flat for white noise; and one Gaussian-process sample rebuilt from 2, 5, 12 and all 128 terms of its expansion
(a) Samples of a Gaussian process with a squared-exponential kernel are smooth random curves. (b) White noise draws an independent value at each grid point. (c) The eigenvalues of the covariance decay quickly for a GP, faster for a longer length scale, and are all equal for white noise, whose total variance therefore grows with the grid. (d) The Karhunen–Loève expansion of one GP sample: the first few eigenfunctions carry the large-scale shape, and a dozen terms already reproduce the sample.

In finite dimensions any two non-degenerate Gaussians have densities with respect to each other, but in infinite dimensions this fails dramatically, for by the Feldman–Hájek theorem two Gaussian measures on a Hilbert space are either equivalent, each absolutely continuous with respect to the other, or mutually singular, living on disjoint sets, with nothing in between. The reason shows up already in high dimensions: under N(0,Id)\mathcal{N}(0, I_d) the normalized squared length ∥x∥2/d\|x\|^2/d concentrates at 1, under N(0,1.21 Id)\mathcal{N}(0, 1.21\, I_d) at 1.21, and as dd grows the overlap vanishes, so that in infinite dimensions scaling the covariance by any constant other than 1 produces a singular measure.

The Cameron–Martin theorem treats shifts. Moving N(0,C)\mathcal{N}(0, C) by a function hh gives an equivalent measure exactly when hh belongs to the Cameron–Martin space C1/2(H)C^{1/2}(\mathcal{H}), the shifts with a finite Cameron–Martin norm ∑nhn2/λn\sum_n h_n^2 / \lambda_n, where hn=⟨h,en⟩h_n = \langle h, e_n \rangle, and the density is then exp⁡{h^(x)−12∥h∥CM2}\exp\{\hat h(x) - \tfrac12 \|h\|^2_{\mathrm{CM}}\}, where h^\hat h is a linear functional determined by hh. For a Brownian bridge, the Cameron–Martin space consists of the functions that vanish at both ends and have a square-integrable derivative, so that a smooth shift is allowed and a rough one is not. Samples of the measure itself almost surely lie outside its Cameron–Martin space, since Brownian paths have no derivative at all, so a sample of the noise is never an allowed shift.

Four panels: distributions of normalized squared length under two Gaussians that separate as dimension grows from 10 to 1000; total variation distance between them rising to 1; a smooth and a rough shift of a Brownian bridge; and the Cameron–Martin norm of each shift, converging for the smooth one and growing without bound for the rough one
(a) The distribution of the normalized squared length under two Gaussians whose standard deviations differ by 10%, in dimensions 10, 100 and 1000. (b) The total variation distance between them tends to 1: they become perfectly distinguishable, the finite-dimensional shadow of mutual singularity. (c) A smooth and a rough shift of a Brownian bridge. (d) The Cameron–Martin norm, accumulated over modes, converges for the smooth shift, whose shifted measure is equivalent to the original, and diverges for the rough one, which is singular.

Three consequences recur in the rest of the post. Many tools of optimal transport in Rd\mathbb{R}^d use densities with respect to volume, which are not available on function spaces, so we will need a formulation that works with samples and costs alone. FFM’s training loss is justified by an absolute-continuity condition, which requires that at each time tt each conditional path distribution be absolutely continuous with respect to the mixture of all of them; a sufficient condition is that the data lie in the Cameron–Martin space of the prior, with σmin⁡>0\sigma_{\min} > 0, and when the data are rougher than the prior this degrades, which is one reason to match the roughness of the base measure to the data, and why white noise is selected on the Navier–Stokes family. And kFFM changes the pairing but inherits this structure, which, as we will see, requires the entropic regularization of optimal transport.

Flow matching, from vectors to functions

We write μ0\mu_0 for the noise or base distribution, ν\nu or μ1\mu_1 for the data distribution, f0f_0 for a noise sample and f1f_1 for a data sample, and a coupling π\pi of μ0\mu_0 and μ1\mu_1 is a joint distribution of pairs (f0,f1)(f_0, f_1) whose two sides are μ0\mu_0 and μ1\mu_1. In Rd\mathbb{R}^d, flow matching learns a velocity field vθ(x,t)v_\theta(x, t), a neural network that says, at every location xx and time t∈[0,1]t \in [0,1], in which direction and how fast a point should move, and to generate a sample one draws noise x0x_0 and follows the velocity field from t=0t = 0 to t=1t = 1,

dxdt=vθ(x,t),x(0)=x0. \frac{dx}{dt} = v_\theta(x, t), \qquad x(0) = x_0 .

Training is simple: take a noise sample x0x_0 and a data sample x1x_1, connect them by a straight line, xt=(1−t) x0+t x1x_t = (1-t)\,x_0 + t\,x_1, and teach the network that at the point xtx_t the velocity should be x1−x0x_1 - x_0,

L(θ)=E ∥vθ(xt,t)−(x1−x0)∥2. \mathcal{L}(\theta) = \mathbb{E}\,\big\| v_\theta(x_t, t) - (x_1 - x_0) \big\|^2 .

Which noise sample goes with which data sample? The standard answer is any, at random, which is the independent coupling π=μ0⊗μ1\pi = \mu_0 \otimes \mu_1, but flow-matching theory says that the learned flow carries the noise distribution to the data distribution for any coupling, so we are free to choose a better one. Random pairing has a cost: straight lines heading to very different data points pass through the same location at the same time, the network can output only one velocity there, and since the minimizer of a squared loss is the average of the targets, it learns a compromise that points toward none of them. Optimal-transport pairing, introduced for flow matching by Tong and colleagues and by Pooladian and colleagues, pairs each noise point with a nearby data point instead, so that the lines stop crossing and the targets at each location agree.

Four panels of two-dimensional points: noise in blue connected by gray lines to data in two orange clusters, crossing everywhere under random pairing and much less under optimal-transport pairing, each beside a field of velocity arrows at time 0.3 colored by which cluster they head to, overlapping in the middle under random pairing and separated under optimal-transport pairing
Flow matching in 2D, with standard Gaussian noise (blue) and data in two clusters (orange). (a) Random pairing: lines from noise to data cross everywhere. (b) The velocity targets at t=0.3t = 0.3, colored by the cluster they head to: in the middle, arrows to the left and to the right overlap, so the network receives conflicting targets at the same place. (c, d) Optimal-transport pairing: nearby noise goes to nearby data, and the two groups of targets separate.

Functional Flow Matching, by Kerrigan and colleagues, carries this recipe over to functions using Gaussian measures. The noise is a Gaussian measure μ0=N(0,C0)\mu_0 = \mathcal{N}(0, C_0), and for each data function ff a Gaussian bridge moves the noise toward ff,

μtf=N(tf,  σt2 C0),σt=1−(1−σmin⁡) t, \begin{gathered} \mu_t^f = \mathcal{N}\big(t f,\; \sigma_t^2\, C_0\big), \\ \sigma_t = 1 - (1 - \sigma_{\min})\, t , \end{gathered}

so that μ0f\mu_0^f is the prior and μ1f\mu_1^f is concentrated around ff. Averaging the bridges over the data, μt=∫μtf dν(f)\mu_t = \int \mu_t^f\, d\nu(f), gives a path of distributions from the prior to, essentially, the data. A sample of the bridge is g=tf+σtξg = t f + \sigma_t \xi with ξ∼μ0\xi \sim \mu_0, and its velocity is

vtf(g)=σt′σt (g−tf)+f=f−(1−σmin⁡) ξ. \begin{aligned} v_t^f(g) &= \frac{\sigma_t'}{\sigma_t}\,\big(g - t f\big) + f \\ &= f - (1 - \sigma_{\min})\,\xi . \end{aligned}

The network is trained to match this conditional velocity with the loss E ∥vtf(g)−vθ(g,t)∥2\mathbb{E}\,\|v_t^f(g) - v_\theta(g, t)\|^2, and the absolute-continuity condition above is what makes this equivalent to matching the intractable velocity of the averaged path. Within each bridge, the path is already the optimal transport between the two Gaussians μ0\mu_0 and μ1f\mu_1^f; what remains random is which data function each noise sample ξ\xi is bridged to, since the endpoint pairing is independent.

Four snapshots of random curves along a bridge toward one data curve, spread widely at time 0 and gathering around the dashed mean curve until, at time 0.95, all lie on it
The FFM bridge toward one data curve ff (dashed: the mean tft f). At t=0t = 0 the samples are GP noise; as tt grows, the mean moves to ff and the spread σt\sigma_t shrinks, so that at t=0.95t = 0.95 every sample is close to ff. With σmin⁡=10−4\sigma_{\min} = 10^{-4}, as in FFM, the bridge ends essentially at ff.

The velocity field takes a function and returns a function, so the network is a neural operator. FFM uses a Fourier Neural Operator (FNO), whose layers act on the Fourier coefficients of their input, each computing σ(Wv+F−1(R⋅Fv))\sigma\big(W v + \mathcal{F}^{-1}(R \cdot \mathcal{F} v)\big), where F\mathcal{F} is the Fourier transform and RR holds learned weights for the lowest modes only. Because the parameters are attached to Fourier modes rather than to grid points, the same network can be evaluated on a finer grid than the one it was trained on.

The problem: random pairs make a hard regression

Writing the bridge with an explicit noise partner shows where the difficulty lies. kFFM uses the same path as FFM, gt=tf1+σtf0g_t = t f_1 + \sigma_t f_0, with the pair (f0,f1)(f_0, f_1) drawn from a coupling π\pi and the target velocity f1−(1−σmin⁡)f0f_1 - (1 - \sigma_{\min}) f_0, which for the independent coupling is exactly FFM. Up to the tiny σmin⁡\sigma_{\min} term, the network regresses the residual u=f1−f0u = f_1 - f_0 between a data function and its noise partner, and under random pairing this residual is the difference of two unrelated draws, a large target that varies a lot from pair to pair, so that the network has to discover the shared structure of the dataset and match each particular instance at the same time. The fix suggested by the two-dimensional picture is to pair each data function with a noise function that is already close to it in a meaningful sense, so that the bridge mostly transports the remaining difference, and the cover figure at the top of this post shows the effect on a synthetic dataset, where the kFFM coupling makes the residuals about 21% smaller on average.

In finite dimensions OT pairing is well established, but in function space it meets two obstacles. The first, as we have seen, is that OT tools relying on densities are unavailable. The second is the choice of distance. The obvious shortcut is to flatten each discretized function into a long vector and use the ordinary squared distance, which is what a direct port of finite-dimensional OT flow matching does and is one of our baselines, CFM-OT(L2L^2), but a flat distance treats grid values as unrelated coordinates and does not see smoothness, derivatives or path roughness, which are exactly what distinguish one function-valued sample from another. The next four sections develop the tools that address both obstacles: optimal transport computed from samples, kernels that encode the geometry, the divergence that combines them, and the signature kernel for paths.

Optimal transport

Given two distributions μ\mu and ν\nu and a cost c(x,y)c(x, y) of moving xx to yy, Monge asked in 1781 for the map TT that pushes μ\mu onto ν\nu, written T#μ=νT_\# \mu = \nu and meaning that if X∼μX \sim \mu then T(X)∼νT(X) \sim \nu, at the least average cost ∫c(x,T(x)) dμ(x)\int c(x, T(x))\, d\mu(x). On the real line with a convex cost the answer is to match quantiles, T=Fν−1∘FμT = F_\nu^{-1} \circ F_\mu, where FF denotes a cumulative distribution function, and the optimal map never crosses itself.

A map cannot split mass, however: if μ\mu puts mass 1/21/2 on a single point and ν\nu spreads mass over three, no map works. Kantorovich replaced maps by couplings, joint distributions π\pi on pairs (x,y)(x, y) whose two sides are μ\mu and ν\nu, and posed

OTc(μ,ν)=inf⁡π∈Π(μ,ν)∫c(x,y) dπ(x,y). \mathsf{OT}_c(\mu, \nu) = \inf_{\pi \in \Pi(\mu, \nu)} \int c(x, y)\, d\pi(x, y).

With c=dpc = d^p for a metric dd, Wp=OTdp1/pW_p = \mathsf{OT}_{d^p}^{1/p} is the Wasserstein distance, a metric on distributions, and the quadratic case W22W_2^2 is the reference our error decomposition compares against. An optimal coupling exists whenever the spaces are Polish, that is, complete and separable metric spaces, and the cost is lower semicontinuous and bounded below; separable Hilbert spaces and the path spaces we use are both Polish, so the problem is well defined on them. The problem also has a dual form, OTc=sup⁡{∫φ dμ+∫ψ dν:φ(x)+ψ(y)≤c(x,y)}\mathsf{OT}_c = \sup \{\int \varphi\, d\mu + \int \psi\, d\nu : \varphi(x) + \psi(y) \le c(x, y)\}, whose optimal functions φ,ψ\varphi, \psi are called potentials.

Adding a penalty that keeps the plan spread out,

OTc,ϵ(μ,ν)=inf⁡π∈Π(μ,ν)∫c dπ+ϵ KL(π ∥ μ⊗ν), \begin{aligned} &\mathsf{OT}_{c,\epsilon}(\mu, \nu) \\ &\quad = \inf_{\pi \in \Pi(\mu, \nu)} \int c\, d\pi + \epsilon\, \mathrm{KL}\big(\pi \,\|\, \mu \otimes \nu\big), \end{aligned}

makes the problem strictly convex and gives the optimal plan an explicit form, a density with respect to the independent coupling,

dπd(μ⊗ν)(x,y)=exp⁡ ⁣(φ(x)+ψ(y)−c(x,y)ϵ), \begin{aligned} &\frac{d\pi}{d(\mu \otimes \nu)}(x, y) \\ &\quad = \exp\!\Big(\frac{\varphi(x) + \psi(y) - c(x, y)}{\epsilon}\Big), \end{aligned}

for two potentials φ,ψ\varphi, \psi. The Sinkhorn algorithm finds them by alternately fixing each marginal, setting φ(x)=−ϵlog⁡∫e(ψ(y)−c(x,y))/ϵ dν(y)\varphi(x) = -\epsilon \log \int e^{(\psi(y) - c(x,y))/\epsilon}\, d\nu(y) and then the same for ψ\psi. On a minibatch of bb noise and bb data functions everything becomes a b×bb \times b matrix, the cost matrix Cij=c(f0(i),f1(j))C_{ij} = c(f_0^{(i)}, f_1^{(j)}), the Gibbs kernel Kij=e−Cij/ϵK_{ij} = e^{-C_{ij}/\epsilon}, and a plan π\pi whose rows and columns each sum to 1/b1/b, and Sinkhorn alternately rescales the rows and columns of KK. The regularization ϵ\epsilon sets how soft the plan is, and training pairs are then sampled from the plan.

Three panels: a source distribution mapped to a two-humped target by quantile matching with non-crossing lines; a plan in which two source atoms of mass one half split their mass among three target atoms of mass one third; and Sinkhorn marginal error falling slowly with a raw squared cost and very quickly with a bounded RBF cost
(a) Monge’s problem on the line: the optimal map matches quantiles, so the lines never cross. (b) Kantorovich’s relaxation: here the source has two atoms of mass 1/2 and the target three of mass 1/3, so each source atom must split, and the optimal plan sends 1/3 straight down and 1/6 to the middle. (c) Sinkhorn on a batch of 64 bumps, 40% of which carry heavy-tailed bursts (ϵ=0.05\epsilon = 0.05, median-normalized costs): with the unbounded L2L^2 cost the marginal error decreases slowly, while with the bounded RBF cost it converges to high precision in about a tenth of the iterations.

The right-hand panel previews a point that becomes our first theorem. When a few pairs have enormous costs, the entries of KK span many orders of magnitude and Sinkhorn converges slowly, whereas a bounded cost keeps KK bounded away from 0 and 1, and convergence is fast regardless of outliers.

Four heat maps on a batch of 12 noise and 12 data points: the cost matrix, and the entropic plans at regularization 1, 0.05 and 0.005, going from nearly uniform to nearly one-to-one along the diagonal
Entropic optimal transport on a batch of 12 noise and 12 data points (1D, sorted, so good matches lie near the diagonal). (a) The cost matrix. (b) With a large ϵ\epsilon the plan is nearly uniform, close to random pairing. (c) At the training default ϵ=0.05\epsilon = 0.05, each noise sample spreads its mass over a few similar partners. (d) With a very small ϵ\epsilon the plan approaches a one-to-one assignment.

Two facts make minibatch entropic OT practical in function space. It needs only the cost matrix, since Sinkhorn never asks for a density, only for the pairwise costs, which sidesteps the first obstacle and leaves the choice of cost. And minibatch plans are a valid coupling: averaged over batches, they define a coupling of the true noise and data distributions, as Fatras and colleagues showed in their analysis of these “expected minibatch plans,” so the flow-matching guarantee still applies, the batch size changes only which pairs are used, and as it grows the plan approaches the full entropic OT plan. In practice a few choices matter for stability, and we recommend them as defaults: dividing each batch’s cost matrix by its median, the most important one since it makes ϵ\epsilon scale-free, running Sinkhorn in the log domain, using ϵ=0.05\epsilon = 0.05, and sampling pairs from the plan rather than averaging over partners.

Because of the entropy term, OTc,ϵ(μ,μ)≠0\mathsf{OT}_{c,\epsilon}(\mu, \mu) \ne 0, so a distribution is not at zero distance from itself. The Sinkhorn divergence removes this bias,

Sc,ϵ(μ,ν)=OTc,ϵ(μ,ν)−12 OTc,ϵ(μ,μ)−12 OTc,ϵ(ν,ν), \begin{aligned} S_{c,\epsilon}(\mu, \nu) &= \mathsf{OT}_{c,\epsilon}(\mu, \nu) \\ &\quad - \tfrac12\, \mathsf{OT}_{c,\epsilon}(\mu, \mu) \\ &\quad - \tfrac12\, \mathsf{OT}_{c,\epsilon}(\nu, \nu), \end{aligned}

and, as Feydy and colleagues showed, it interpolates between optimal transport as ϵ→0\epsilon \to 0 and a kernel discrepancy, an MMD, as ϵ→∞\epsilon \to \infty. We will see what this interpolation looks like with a kernel cost once kernels are in place.

Kernels, RKHS and MMD

A positive-definite kernel κ(x,y)\kappa(x, y) is a symmetric similarity score whose matrices [κ(xi,xj)]ij[\kappa(x_i, x_j)]_{ij} are positive semidefinite for every finite set of points, and every such kernel comes with a Hilbert space of functions, its reproducing kernel Hilbert space (RKHS) Hκ\mathcal{H}_\kappa, in which evaluating a function at a point is a continuous operation. By the Riesz representation theorem, evaluation at xx is an inner product with a special element κ(x,⋅)\kappa(x, \cdot),

f(x)=⟨f,κ(x,⋅)⟩Hκ,κ(x,y)=⟨κ(x,⋅),κ(y,⋅)⟩Hκ. \begin{gathered} f(x) = \langle f, \kappa(x, \cdot) \rangle_{\mathcal{H}_\kappa}, \\ \kappa(x, y) = \langle \kappa(x, \cdot), \kappa(y, \cdot) \rangle_{\mathcal{H}_\kappa}. \end{gathered}

The map ϕ(x)=κ(x,⋅)\phi(x) = \kappa(x, \cdot) is the feature map, which turns each point into a function, and inner products of features are kernel evaluations, so the features never need to be written down, which is the “kernel trick.”

Averaging the features gives the kernel mean embedding of a distribution, mμ=EX∼μ κ(X,⋅)m_\mu = \mathbb{E}_{X \sim \mu}\, \kappa(X, \cdot), a single element of the RKHS, which exists for every distribution exactly when the kernel is bounded, one more reason boundedness will matter. The distance between embeddings is the maximum mean discrepancy (MMD),

MMD2(μ,ν)=∥mμ−mν∥Hκ2=E κ(X,X′)+E κ(Y,Y′)−2 E κ(X,Y), \begin{aligned} \mathrm{MMD}^2(\mu, \nu) &= \|m_\mu - m_\nu\|^2_{\mathcal{H}_\kappa} \\ &= \mathbb{E}\,\kappa(X, X') + \mathbb{E}\,\kappa(Y, Y') \\ &\quad - 2\,\mathbb{E}\,\kappa(X, Y), \end{aligned}

with X,X′∼μX, X' \sim \mu and Y,Y′∼νY, Y' \sim \nu independent. Equivalently, MMD(μ,ν)=sup⁡∥h∥H≤1E h(X)−E h(Y)\mathrm{MMD}(\mu, \nu) = \sup_{\|h\|_{\mathcal{H}} \le 1} \mathbb{E}\,h(X) - \mathbb{E}\,h(Y), the largest difference in expectation over smooth test functions, whose best test function, the witness, is proportional to mμ−mνm_\mu - m_\nu. MMD needs only samples, which is why it is our main evaluation metric, and its unbiased estimator drops the terms where a sample is compared with itself, so it can be slightly negative when two distributions are close, as some values in our results are.

MMD is a true distance between distributions only if the embedding is injective, that is, if mμ=mνm_\mu = m_\nu implies μ=ν\mu = \nu, and such kernels are called characteristic. A linear kernel only compares means, and a quadratic one means and variances, while the Gaussian or RBF kernel is characteristic on every separable Hilbert space, as Ziegel and colleagues showed, so the RBF kernels we use on L2L^2 and on Sobolev spaces are characteristic. Universal kernels, whose RKHS can approximate every continuous function on a compact set, are characteristic too, and the signature kernel is universal on compact sets of time-augmented paths.

Three panels: many small bumps, one per sample, averaging into two mean-embedding curves for a Gaussian and a two-bump distribution; the witness function, positive where the Gaussian has more mass and negative where the two-bump distribution does; and bar heights of squared MMD, zero for linear and quadratic kernels and positive for the RBF kernel
Kernel mean embeddings with an RBF kernel, for a Gaussian and a two-bump distribution with the same mean and variance. (a) Each sample becomes a bump (thin lines), and their averages are the mean embeddings (thick lines). (b) The witness function, positive where the Gaussian has more mass and negative where the two-bump distribution does. (c) The squared MMD under three kernels: the linear and quadratic kernels see only the first two moments, which are equal, so their MMD is exactly 0, while the RBF kernel is characteristic and tells the distributions apart.

Every kernel defines a squared distance between points, the squared distance between their features,

cκ(x,y)=∥ϕ(x)−ϕ(y)∥Hκ2=κ(x,x)+κ(y,y)−2 κ(x,y), \begin{aligned} c_\kappa(x, y) &= \|\phi(x) - \phi(y)\|^2_{\mathcal{H}_\kappa} \\ &= \kappa(x, x) + \kappa(y, y) - 2\,\kappa(x, y), \end{aligned}

and the plain inner product κ(f,g)=⟨f,g⟩\kappa(f, g) = \langle f, g \rangle gives back the L2L^2 cost. kFFM uses three kernels on function spaces. The Euclidean RBF, κ(f,g)=exp⁡(−∥f−g∥2/2σ2)\kappa(f, g) = \exp(-\|f - g\|^2 / 2\sigma^2), gives cκ=2 (1−e−∥f−g∥2/2σ2)c_\kappa = 2\,(1 - e^{-\|f - g\|^2 / 2\sigma^2}), which for nearby pairs is approximately ∥f−g∥2/σ2\|f - g\|^2 / \sigma^2 and so keeps the L2L^2 ranking, while for distant pairs it levels off at 2, so that outliers cannot dominate; its bandwidth σ\sigma is set by the median heuristic. The Sobolev RBF is the same kernel with the L2L^2 norm replaced by the Sobolev norm HkH^k, which also compares derivatives, so that fast wiggles count. And the third is the signature kernel for paths, to which we come shortly.

Three panels: the squared cost growing without bound with distance while two RBF costs saturate; a curve with a slightly shifted copy and a copy with fast wiggles; and bars showing equal squared distances of 0.16 for both changes but Sobolev distances of 0.99 and 13.65
(a) The L2L^2 cost grows without bound with the distance between two functions, while the RBF cost agrees with it for nearby pairs and saturates for distant ones, a larger bandwidth saturating later. (b) Two changes of ff with the same L2L^2 distance, a small shift and a fast wiggle. (c) Their L2L^2 distances are equal (0.16), but the Sobolev distance, which also compares derivatives, is 14 times larger for the wiggle.

The Hilbert Sinkhorn Divergence

Li and colleagues combined the two preceding ideas, embedding both distributions in the RKHS with the feature map, ϕ#μ\phi_\# \mu and ϕ#ν\phi_\# \nu, and comparing the embedded distributions with entropic optimal transport under the squared RKHS distance,

OTϵHκ(ϕ#μ,ϕ#ν)=inf⁡πϕ∫∥u−v∥Hκ2 dπϕ(u,v)+ϵ KL(πϕ ∥ ϕ#μ⊗ϕ#ν). \begin{aligned} &\mathsf{OT}^{\mathcal{H}_\kappa}_{\epsilon}(\phi_\#\mu, \phi_\#\nu) \\ &\quad = \inf_{\pi_\phi} \int \|u - v\|^2_{\mathcal{H}_\kappa}\, d\pi_\phi(u, v) \\ &\qquad + \epsilon\, \mathrm{KL}\big(\pi_\phi \,\|\, \phi_\#\mu \otimes \phi_\#\nu\big). \end{aligned}

The Hilbert Sinkhorn Divergence (HSD) is its debiased version, Sϵ=OTϵ(μ,ν)−12OTϵ(μ,μ)−12OTϵ(ν,ν)S_\epsilon = \mathsf{OT}_\epsilon(\mu,\nu) - \tfrac12 \mathsf{OT}_\epsilon(\mu,\mu) - \tfrac12 \mathsf{OT}_\epsilon(\nu,\nu), computed in the RKHS. The features may be infinite-dimensional, but by the kernel trick the whole problem can be solved on the original space with the kernel-induced cost cκc_\kappa, since the optimal plans correspond to each other under the feature map and the divergences are equal, so that the HSD is entropic OT under the cost cκc_\kappa, and its plan is exactly the coupling kFFM uses.

The interpolation between OT and MMD takes a clean form here. As ϵ→0\epsilon \to 0, the HSD tends to optimal transport between the embedded distributions, kernel OT; as ϵ→∞\epsilon \to \infty, the plan tends to the independent coupling, and the debiased divergence becomes

E cκ(X,Y)−12 E cκ(X,X′)−12 E cκ(Y,Y′)=E κ(X,X′)+E κ(Y,Y′)−2 E κ(X,Y)=MMDκ2(μ,ν), \begin{aligned} &\mathbb{E}\,c_\kappa(X, Y) - \tfrac12\,\mathbb{E}\,c_\kappa(X, X') \\ &\quad - \tfrac12\,\mathbb{E}\,c_\kappa(Y, Y') \\ &= \mathbb{E}\,\kappa(X, X') + \mathbb{E}\,\kappa(Y, Y') \\ &\quad - 2\,\mathbb{E}\,\kappa(X, Y) = \mathrm{MMD}^2_\kappa(\mu, \nu), \end{aligned}

because the κ(x,x)\kappa(x, x) terms cancel. Both ends are genuine distances between distributions when the kernel is characteristic, whereas with the plain squared cost the large-ϵ\epsilon end is only the squared difference of the means, which is blind to everything else.

Two panels plotting the Sinkhorn divergence between two samples against the regularization: with the squared cost it falls from the Wasserstein value to zero, and with an RBF kernel cost it falls from kernel optimal transport to a positive squared MMD
The Sinkhorn divergence between two samples with the same mean and variance (a Gaussian and a two-bump distribution, 120 points each), as the regularization varies; the dashed lines are the exact limits. (a) With the L2L^2 cost it falls from W22W_2^2 at small ϵ\epsilon to 0 at large ϵ\epsilon, where it only compares means. (b) With an RBF kernel cost, the Hilbert Sinkhorn divergence, it moves from kernel OT at small ϵ\epsilon to the squared MMD with the same kernel at large ϵ\epsilon, which still separates the two distributions.

In our paper the HSD is the object of analysis, what the theorems control, while training uses only its plan, since the two debiasing terms do not depend on the coupling and cannot change which plan is optimal. Li and colleagues established its statistical consistency and finite-sample bounds for vector data, and our theorems extend the setting to function spaces.

Paths and the signature kernel

Time series are paths, and paths have structure that neither L2L^2 nor Sobolev norms capture well, namely the order in which things happen and how rough the path is. A path is a continuous map x:[0,T]→Rdx : [0, T] \to \mathbb{R}^d, and its roughness is measured by its pp-variation, ∥x∥p-var=(sup⁡∑i∣xti+1−xti∣p)1/p\|x\|_{p\text{-var}} = \big(\sup \sum_i |x_{t_{i+1}} - x_{t_i}|^p\big)^{1/p}, the supremum over all partitions of [0,T][0, T]. Paths of finite pp-variation form a Banach space; we work with p∈[1,2)p \in [1, 2), for which the integrals below are classical Young integrals, and treat a discrete time series as the piecewise-linear path through its observations.

The signature of a path is the sequence of its iterated integrals,

S(x)=(1, S(1)(x), S(2)(x), …),S(m)(x)=∫0<t1<⋯<tm<Tdxt1⊗⋯⊗dxtm, \begin{gathered} S(x) = \big(1,\, S^{(1)}(x),\, S^{(2)}(x),\, \dots\big), \\ S^{(m)}(x) = \int\limits_{0 < t_1 < \dots < t_m < T} dx_{t_1} \otimes \dots \otimes dx_{t_m}, \end{gathered}

where S(m)(x)S^{(m)}(x) lies in (Rd)⊗m(\mathbb{R}^d)^{\otimes m}. The first level is the increment, S(1)=xT−x0S^{(1)} = x_T - x_0, and the second is a d×dd \times d matrix whose symmetric part is fixed by the increment, 12S(1)⊗S(1)\tfrac12 S^{(1)} \otimes S^{(1)}, and whose antisymmetric part is the Lévy area, 12(Sij−Sji)\tfrac12 (S^{ij} - S^{ji}), the signed area between the path and the straight chord from its start to its end, while higher levels record higher-order interactions between the coordinates. Three properties make the signature a good feature map for paths. It ignores speed and position and sees shape and order, for it does not change if the path is traversed faster or slower or is translated; without extra information it identifies a path only up to “tree-like” detours that retrace themselves, but adding time as a coordinate, xˉ(t)=(t,x(t))\bar x(t) = (t, x(t)), removes this ambiguity and makes the signature sensitive to timing, and we always use this time augmentation. It decays factorially, ∥S(m)(x)∥≤Lm/m!\|S^{(m)}(x)\| \le L^m / m!, where LL is the length of the path, so truncating at a moderate level loses little. And it is universal, since on compact sets of time-augmented paths linear functions of the signature approximate every continuous function of the path, just as polynomials approximate continuous functions on an interval. The signature of a concatenation of two paths is the tensor product of their signatures, a fact called Chen’s identity, and it is how the signatures in the figures below were computed, one linear segment at a time.

Three panels: a two-dimensional path with its chord, the area between them shaded blue where negative and orange where positive; bars of six level-1 and level-2 signature terms for a path, the same path at a different speed, and a path with the same endpoints but a different shape; and the size of each signature level falling below a factorial bound
(a) A 2D path (gray) and its first two signature levels: the increment (arrow) and the Lévy area, the signed area between the path and its chord, the blue region traversed clockwise and counted negative, the orange one counterclockwise and counted positive. (b) The six level-1 and level-2 terms for a path A, for the same curve traversed at a different speed (identical), and for a path B with the same endpoints but a different shape: B shares A’s increment and the symmetric terms, which the increment determines, and only the area terms differ. (c) The size of the level-mm term of A falls below the factorial bound Lm/m!L^m/m!.

The signature kernel is the inner product of two signatures over all levels,

κsig(x,y)=⟨S(x),S(y)⟩=∑m≥0⟨S(m)(x),S(m)(y)⟩, \begin{aligned} \kappa_{\mathrm{sig}}(x, y) &= \langle S(x), S(y) \rangle \\ &= \sum_{m \ge 0} \langle S^{(m)}(x), S^{(m)}(y) \rangle , \end{aligned}

and computing it does not require forming the signatures, whose size grows like dmd^m. Salvi and colleagues showed that K(s,t)=⟨S(x[0,s]),S(y[0,t])⟩K(s, t) = \langle S(x_{[0,s]}), S(y_{[0,t]}) \rangle, the kernel between the two paths stopped at times ss and tt, solves

K(s,t)=1+∫0s ⁣ ⁣∫0tK(u,v) ⟨dxu,dyv⟩, K(s, t) = 1 + \int_0^s \!\! \int_0^t K(u, v)\, \langle dx_u, dy_v \rangle ,

which for smooth paths is the hyperbolic Goursat PDE ∂s∂tK=⟨x˙s,y˙t⟩ K\partial_s \partial_t K = \langle \dot x_s, \dot y_t \rangle\, K with K=1K = 1 on the two axes. The kernel is K(T,T)K(T, T), obtained with a standard finite-difference solver, and we use the pySigLib library. Two hyperparameters of this computation appear in our sensitivity study, the refinement of the PDE grid, called the dyadic order, and the bandwidth of a static kernel through which the increments can first be passed, and since the solver’s grid has one cell per pair of time steps, memory grows with the product of the two path lengths.

For a one-dimensional series, the lead-lag transform pairs the path with a copy of itself delayed by one step, and the Lévy area of the resulting 2D path equals half the sum of squared increments, the path’s quadratic variation, which measures its volatility. This is how the signature kernel tells a calm path from a jittery one even when their overall trends match, something the L2L^2 distance cannot do, as the toy examples below show.

Four panels: two time-augmented paths; a heat map of the signature-kernel PDE solution over the two time axes; truncated level-by-level sums converging to the PDE value; and Lévy areas of lead-lag Brownian paths following a curve proportional to the square of volatility
(a) Two time-augmented paths. (b) The solution K(s,t)K(s, t) of the signature-kernel PDE, whose value at (1,1)(1, 1) is the signature kernel. (c) The truncated sum of level-by-level inner products, computed from explicit signatures, converges to the PDE value (2.2626 for both). (d) For Brownian paths vW(t)v W(t) with volatility vv, the Lévy area of the lead-lag path follows v2/2v^2/2, half the quadratic variation.

kFFM on one page

kFFM is FFM with one step changed. Each training iteration draws a batch of bb noise functions f0f_0, from a GP or white noise, and bb data functions f1f_1, and computes the b×bb \times b kernel cost matrix

Cij=κ(f0(i),f0(i))+κ(f1(j),f1(j))−2 κ(f0(i),f1(j)), \begin{aligned} C_{ij} &= \kappa(f_0^{(i)}, f_0^{(i)}) + \kappa(f_1^{(j)}, f_1^{(j)}) \\ &\quad - 2\,\kappa(f_0^{(i)}, f_1^{(j)}), \end{aligned}

which it divides by its median. It then runs log-domain Sinkhorn with ϵ=0.05\epsilon = 0.05 to get the plan π\pi, samples bb pairs (f0,f1)(f_0, f_1) from π\pi, draws a time tt uniformly in [0,1][0, 1] and forms g=t f1+σt f0g = t\,f_1 + \sigma_t\,f_0, and finally regresses the network output vθ(g,t)v_\theta(g, t) onto the target f1−(1−σmin⁡) f0f_1 - (1 - \sigma_{\min})\,f_0 with a squared loss and takes a gradient step.

The kernel enters at exactly one place, the cost matrix. No sample is mapped into the RKHS: the neural operator sees the original discretized functions, the paths are built in the original space, and generation is unchanged, starting from noise and following the learned velocity, so that if the plan is replaced by random pairing the algorithm is exactly FFM. The kernel should match the data class, the signature kernel for path-like data, the Sobolev or Euclidean RBF for smooth fields, where the Euclidean RBF is a reasonable default, and RBF or L2L^2 costs for very long paths, where the memory of the signature kernel becomes the bottleneck.

Two toy examples where flat pairing fails

Before the benchmarks, we ran a controlled test of the two weaknesses of the L2L^2 cost, that it is unbounded and that it is blind to structure. We draw sources and targets that have an intended matching, sort both so that the intended matches lie on the diagonal, compute the entropic plan under each cost exactly as in training, with b=64b = 64 and median-normalized costs, and report how much of the plan’s mass falls on the intended matches, where a uniform plan scores about 0.14 to 0.19.

Four heat maps of transport plans with sources and targets sorted so that intended matches lie on the diagonal: for bumps with random bursts, the squared cost gives a scattered plan and the RBF cost a clear diagonal; for Brownian paths of different volatility, the squared cost gives a scattered plan and the signature kernel a diagonal band
Four of the twelve panels of the paper’s toy-coupling figure. Each panel is a plan, with rows for sources and columns for targets, both sorted so that the intended matches are on the diagonal; bright means more mass. Top: bumps matched by location, where 40% of the samples carry large, independent bursts. Bottom: Brownian paths f=vW(t)f = v W(t) with volatility vv between 0.5 and 2, matched by volatility.

The top row shows a conditioning failure. The burst pairs dominate the median normalization of the unbounded L2L^2 cost, the contrasts between clean pairs fall below ϵ\epsilon, and the plan is close to uniform, with 0.16 of its mass on the intended matches, while the bounded RBF cost saturates the bursts and recovers the matching (0.58) whatever the burst amplitude, and any bounded cost would fix this. The bottom row shows a geometry failure. For two independent Brownian paths with volatilities vv and ww, the expected L2L^2 cost is (v2+w2)∫t dt(v^2 + w^2)\int t\,dt, which does not depend on whether vv and ww are similar, so it carries no signal about the intended matching and the plan stays close to uniform (0.25), whereas the signature kernel reads volatility through the quadratic variation and concentrates on the intended matching (0.43), and only the change of geometry fixes this. The full figure in the paper also shows that no single cost wins every scenario, since the signature kernel, for example, fails on the burst example, which is why kFFM chooses the kernel per data class.

What the theory says, and what it does not

With the background in place, the three theorems can be stated almost as in the paper. They certify the kernel pairing; they do not, by themselves, explain why sample quality improves, which is an empirical claim tested in the results.

Theorem 1, on well-posedness and uniform boundedness, states that if X\mathcal{X} is a real Banach space and κ\kappa a bounded positive-definite kernel on it, with κ(f,f)≤B\kappa(f, f) \le B for all ff, then 0≤cκ(f,g)≤4B0 \le c_\kappa(f, g) \le 4B for all f,gf, g, and for every ϵ>0\epsilon > 0 the Hilbert Sinkhorn Divergence is finite for all pairs of distributions, with ∣Sκ,ϵ(μ,ν)∣≤8B|S_{\kappa,\epsilon}(\mu, \nu)| \le 8B. Finiteness alone is not the point, since under a second-moment assumption the L2L^2 cost is integrable too, and for a Gaussian prior E∥f0∥2=tr⁡(C0)<∞\mathbb{E}\|f_0\|^2 = \operatorname{tr}(C_0) < \infty. What the theorem adds is a bound that is uniform, with no assumption on the distributions, and that holds without an inner product. The first part conditions the solver, for the Gibbs kernel e−cκ/ϵe^{-c_\kappa/\epsilon} stays bounded away from 0 and 1 in every batch, so Sinkhorn converges regardless of outlier pairs, and we measured the bounded RBF cost to be about 8 times faster per batch than the unbounded L2L^2 cost at b=512b = 512. The second part covers path spaces, which are Banach but not Hilbert, and therefore licenses the signature kernel, our strongest configuration.

Theorem 2 is an error decomposition. Let X\mathcal{X} be a compact metric space of diameter MM, let W(μ,ν)\mathcal{W}(\mu,\nu) be optimal transport with the quadratic cost d2d^2, and let Δκ=sup⁡x,y∣cκ(x,y)−d(x,y)2∣\Delta_\kappa = \sup_{x,y} |c_\kappa(x, y) - d(x, y)^2| be the largest gap between the kernel cost and the squared distance; then for every ϵ>0\epsilon > 0 and δ∈(0,M]\delta \in (0, M],

∣OTκ,ϵ(μ,ν)−W(μ,ν)∣≤Δκ⏟kernel mismatch+2 ϵlog⁡N(X,δ)⏟entropic blur+8Mδ⏟covering, \begin{aligned} &\big|\mathsf{OT}_{\kappa,\epsilon}(\mu,\nu) - \mathcal{W}(\mu,\nu)\big| \\ &\quad \le \underbrace{\Delta_\kappa}_{\text{kernel mismatch}} + \underbrace{2\,\epsilon \log \mathcal{N}(\mathcal{X}, \delta)}_{\text{entropic blur}} \\ &\qquad + \underbrace{8M\delta}_{\text{covering}} , \end{aligned}

where N(X,δ)\mathcal{N}(\mathcal{X}, \delta) is the covering number. The compact spaces are the ones built earlier, Sobolev balls {∥f∥Hα≤R}\{\|f\|_{H^\alpha} \le R\} inside HkH^k for α>k\alpha > k, and compact sets of paths for the signature kernel. The entropic and covering terms vanish as ϵ\epsilon and δ\delta go to zero, but the mismatch term is irreducible, since more samples or finer grids do not shrink it. For the RBF kernel it is small at large bandwidth, because cRBFc_{\mathrm{RBF}} is a rescaled L2L^2 cost up to relative error O((D/σ)2)O((D/\sigma)^2) for batch diameter DD, and on smooth Navier–Stokes fields the plans under the RBF and L2L^2 costs nearly coincide, at a total variation of at most 0.02. For the signature kernel the mismatch is large by design, since we want a different notion of closeness, and the theorem then controls the deviation from transport with the chosen geometry rather than proximity to L2L^2 transport; the pair rankings differ substantially from L2L^2, with a rank correlation of 0.33–0.37 on the AEMET and gene-expression data, which is where the largest gains occur. In rough regimes the bound degrades on two fronts, as a smooth kernel cost saturates, enlarging Δκ\Delta_\kappa, and the covering number explodes as the effective smoothness approaches the compactness threshold.

Theorem 3 concerns discretization invariance. On a bounded periodic domain Ω⊂Rd\Omega \subset \mathbb{R}^d, take distributions on the Sobolev ball {∥f∥Hα≤R}\{\|f\|_{H^\alpha} \le R\} with α>k≥0\alpha > k \ge 0, viewed inside HkH^k, and let μN,νN\mu_N, \nu_N be their projections onto the span of about NN Fourier modes; then for the Sobolev-RBF kernel on HkH^k and every ϵ>0\epsilon > 0,

∣KOTϵ(k)(μ,ν)−KOTϵ(k)(μN,νN)∣≤4R2σ2 N−2(α−k)/d, \begin{aligned} &\big|\mathsf{KOT}^{(k)}_\epsilon(\mu, \nu) - \mathsf{KOT}^{(k)}_\epsilon(\mu_N, \nu_N)\big| \\ &\quad \le \frac{4R^2}{\sigma^2}\, N^{-2(\alpha - k)/d}, \end{aligned}

or O(h2(α−k))O(h^{2(\alpha - k)}) in terms of the grid spacing h=N−1/dh = N^{-1/d}. This is the truncation inequality from the section on function spaces carried through the kernel and the transport problem: the smoother the data relative to the kernel’s norm, the faster the objective computed on a grid approaches the continuum one, which is the formal counterpart of resolution invariance. Combining Theorems 2 and 3 bounds the gap between the objective the model actually computes and continuum quadratic optimal transport in HkH^k by four interpretable terms, kernel mismatch, entropic blur, covering and discretization, of which only the last vanishes as the resolution grows.

One more property matters: kFFM inherits FFM’s foundations. FFM’s loss rests on absolute-continuity relations between Gaussian measures, and with a bounded cost the entropic plan has a density with respect to the independent coupling, exp⁡((φ⊕ψ−cκ)/ϵ)\exp((\varphi \oplus \psi - c_\kappa)/\epsilon), with bounded potentials, so this density is bounded above and away from zero. The plan is therefore equivalent to the independent coupling, its conditionals are equivalent to the prior, and every absolute-continuity relation that FFM relies on transfers to kFFM. This is where entropic regularization matters beyond computation, since an unregularized OT plan can be singular with respect to the independent coupling, and the argument would fail.

Finite-dimensional OT flow matching is often motivated by straighter paths and hence cheaper sampling, but we make no straightness or sampling-efficiency claim. kFFM changes only which endpoints are paired, the path between them is the same straight-line template, and the geometry of Gaussian measures on Hilbert spaces, Bures–Wasserstein rather than flat, does not inherit the Euclidean picture behind that intuition, as the paper’s appendix states precisely.

Results

We used four sequence datasets, AEMET (the mean annual temperature profile at 73 Spanish weather stations, 365 points), gene expression, economic series (population, GDP and labor) and volatility paths from the Heston model, and four PDE datasets, the 1D Korteweg–de Vries (KdV) equation, the 2D incompressible Navier–Stokes equation and stochastic versions of both, with a turbulent Navier–Stokes benchmark added at the end. The baselines are FFM; functional diffusion (DDPM); a score-based denoising diffusion operator (DDO with NCSN noise scales); an adversarial neural operator (GANO); and CFM-OT(L2L^2), which uses the same Sinkhorn solver and neural operator as kFFM but applies OT to the flattened grid with the raw L2L^2 cost and white noise.

The main metric is the MMD with an RBF kernel, computed with the unbiased estimator, so that a tiny or negative value means “indistinguishable from the data at this sample size,” and percentage gains there are not meaningful. Because an RBF-based metric could favor an RBF-based pairing, we also report non-kernel metrics (the sliced Wasserstein distance and the marginal W1W_1), structural diagnostics (the autocorrelation error for sequences and the log-spectrum error for PDEs) and physics diagnostics on Navier–Stokes. Results are over 10 seeds (20 for the comparison with FFM on four datasets), and all methods share seeds, so comparisons are paired by seed.

For each dataset, the cost and the base measure of kFFM are chosen on a held-out validation split by sliced Wasserstein distance, a criterion that does not involve the kernels being compared. The selected configurations follow the rule given above: the signature kernel with a GP base on path-like data, except short-horizon Heston, where the top configurations are statistically indistinguishable and the rule picks the RBF cost with white noise; the Sobolev-RBF cost with a GP base on the KdV family; and the Euclidean RBF cost with white noise on the Navier–Stokes family, whose rough fields a smooth GP prior matches poorly.

Distributional quality

kFFM has the lowest MMD of all six methods on all eight datasets. Where the strongest baseline is not at the noise floor, the improvement over it is 74% on stochastic KdV, 60% on Economy and on gene expression, 34% on stochastic Navier–Stokes and 8% on Navier–Stokes, and relative to CFM-OT(L2L^2), the same solver and network with a flat L2L^2 cost, kFFM lowers MMD by 75% on gene expression and 66% on Economy.

Eight small panels, one per dataset, each comparing FFM and kFFM by mean and standard deviation of MMD over seeds, with paired t-test and Wilcoxon p-values, kFFM lower in every panel
kFFM (selected configuration, in parentheses) against FFM: MMD-RBF × 10³, mean ± standard deviation over seeds shared by both methods, with seed-paired t-test and Wilcoxon signed-rank p-values (numbers from the paper’s Table 2). In the panels marked as changing only the pairing, both methods use the same GP noise. In the three panels where the base measure also changes, the selected kFFM uses white noise while FFM keeps its GP prior, and the effect of the kernel cost at that base is isolated by the comparison with CFM-OT(L2L^2) in the next figure.

kFFM improves on FFM on every dataset at the 0.05 level under both tests, and on the four datasets with the largest gaps it wins on every seed. The evidence is not equally strong everywhere, however: on AEMET, Heston, KdV and stochastic KdV the MMD values sit near zero and the p-values are between 0.011 and 0.039, significant but modest.

On the same runs, kFFM also improves the sliced Wasserstein distance and the marginal W1W_1 over FFM on the four datasets of the paper’s non-kernel table, by 18–20% on gene expression and Economy, where each comparison is significant when paired by seed, and by 2.9–4.1× on Navier–Stokes and stochastic Navier–Stokes; for the signature configurations the pairing kernel and the evaluation kernel already differ. The structural diagnostics show what MMD understates, for CFM-OT(L2L^2) matches pointwise values reasonably well but has 105× and 66× larger autocorrelation error than kFFM with the signature kernel on Economy and gene expression, and function-level structure is where the kernel pairing helps most.

Bounded cost or function-space geometry?

Both effects isolated in the toy examples show up on real data, on different data classes.

Two bar charts of MMD: on gene expression and Economy, random pairing, a raw squared cost and a bounded RBF cost all score similarly, while the signature kernel scores far lower; on the two Navier–Stokes datasets, the bounded RBF cost scores lower than or close to the raw squared cost
MMD-RBF × 10³ (means from the paper; lower is better). (a) On the path datasets, pairing with a Euclidean cost, raw or bounded, does no better than random pairing, and only the signature kernel helps (matched runs, all with the same training budget). (b) On the Navier–Stokes family, at the same white-noise base, the bounded RBF cost improves on the raw L2L^2 cost on stochastic Navier–Stokes and matches it on Navier–Stokes.

On paths, geometry is what matters, for no Euclidean cost recovers the gains of the signature kernel: on gene expression, kFFM with the raw L2L^2 cost reaches 20.0 and with the RBF cost 20.7, against 7.11 with the signature kernel, and on Economy 7.55 and 7.60 against 2.89, so the signature kernel genuinely re-ranks pairs relative to L2L^2. On smooth fields, boundedness is what matters, since the RBF cost is a monotone transform of L2L^2, with the same ranking of pairs and the outliers compressed, and on smooth Navier–Stokes fields the plans under the two costs nearly coincide, at a total variation of at most 0.02. What the bounded cost buys there is conditioning, robustness to ϵ\epsilon and faster Sinkhorn, and it matches or improves the unbounded cost at the same base measure.

A turbulent stress test

The Navier–Stokes benchmark above is smooth, the regime where our theory is informative. To test the rough regime, where Theorem 2 is weakest, we used the turbulent 2D Navier–Stokes benchmark of the FNO paper (viscosity 10−510^{-5}, Reynolds number about 2000) and compared seven methods over five seeds, with the same network and training budget for all. Besides MMD and the sliced Wasserstein distance, we measured physics diagnostics: the distributions of vorticity and enstrophy, the skewness and kurtosis of the vorticity, and the energy spectrum.

A grid of vorticity fields in red and blue, four per row, for the data, kFFM, FFM, a score-based diffusion operator and DDPM: kFFM and FFM show large-scale bands like the data, the score-based model near-blank noise, and DDPM coarse blobs
Vorticity samples for turbulent Navier–Stokes, from the paper, on a shared color scale; for each method, the samples whose enstrophy is closest to the data median. kFFM and FFM reproduce the data’s large-scale organization, the score-based DDO/NCSN produces near-noise fields, and DDPM produces coarse blobs.
Two line plots: the pooled distribution of vorticity values on a log scale, bimodal for the data and for kFFM, a narrow spike for the score-based model and unimodal for DDPM; and energy spectra against wavenumber, the flow-matching curves following the data and the score-based curve nearly flat
From the paper. Left: the pooled distribution of vorticity values (log scale): the data distribution is bimodal and heavy-tailed, kFFM tracks it, DDO/NCSN collapses to a narrow spike, and DDPM is unimodal. Right: the energy spectrum: the flow-matching spectra follow the data across scales, while that of DDO/NCSN is nearly flat.

kFFM has the best value on five of the seven metrics and ties for the best MMD, while CFM-OT(L2L^2) at the same white-noise base overlaps with kFFM within one standard deviation. OT-paired flow matching is far more robust than the score-based and diffusion alternatives, with 23–32× better vorticity-distribution W1W_1, 5.7–6.9× better enstrophy W1W_1, and 17–43× smaller kurtosis error. Against FFM with its GP prior, kFFM improves every metric except kurtosis, where the two tie, with seed-paired significance (its MMD, for example, is 73× lower, p=3×10−4p = 3 \times 10^{-4}). This gain combines two changes, however: the three methods that use the GP base are indistinguishable from each other, so in this regime the white-noise base does most of the work, and at that base the kernel and raw L2L^2 costs tie. The turbulent benchmark therefore supports OT-paired flow matching with a white-noise base as the strongest current option, without singling out the kernel cost.

Cost and sensitivity

The per-batch Sinkhorn solve adds 5–7% to end-to-end training time at the largest resolution we train, 128×128128 \times 128 Navier–Stokes. Its cost grows with the square of the batch size but does not depend on the model size or the resolution, so its share shrinks as the network grows; peak memory for the pairing is 80–151 MB, 1.5–9% of the network’s own peak, and one pairing takes about as long as one network step. The bounded RBF cost makes Sinkhorn about 8 times faster than the raw L2L^2 cost at b=512b = 512 (9 against 73 ms per batch) and training 15–25% faster end-to-end, and giving FFM the extra time as additional epochs does not close the gap.

Results are indistinguishable within seed-to-seed variation for RBF bandwidths between 0.1 and 2. With bounded costs they barely move across the Sinkhorn regularization values we swept, whereas the raw L2L^2 cost degrades by about 10× at ϵ≥0.5\epsilon \ge 0.5 on long volatility paths, and they are unchanged across the signature kernel’s dyadic order (0 to 3) and static bandwidth (0.1 to 5). The kernel family is the choice that matters: match the kernel to the data class, set the bandwidth by the median heuristic, and validate with a non-kernel metric on held-out data. The properties the theorems require, boundedness and characteristicness, hold for every kernel we consider, so the theory constrains the family only weakly and the selection rule does the rest.

Limitations

The theory certifies the pairing, not the samples: it establishes well-posedness, an error decomposition and discretization invariance, but says nothing about straightness or sampling speed, on which we make no claim, and the gains in sample quality remain an empirical finding. Rough data are where the theory says least, for on the turbulent benchmark the distances between noise and data functions more than double relative to the standard benchmark (a median of 91 against 40), which pushes the RBF cost into saturation, so that the mismatch term grows and the covering term of Theorem 2 is large for any practical ϵ\epsilon, and there the empirical results carry the method. Turbulence is still open, since the failures sit mainly with the score-based and diffusion baselines, but all methods, ours included, leave headroom (the best sliced Wasserstein distance is 0.19), and in this regime the base measure rather than the pairing does most of the work. The signature kernel is memory-hungry on long paths, its memory growing with path length (0.63 GB at 100 time points against 12.5 GB at 512), so very long paths fall back to RBF or L2L^2 costs. And some gains are modest, for on four datasets the MMD values are near the noise floor and the paired p-values are between 0.011 and 0.039.

Takeaways

Pairing is a design choice: flow matching works with any coupling of noise and data, random pairing is the default rather than a requirement, and pairing each data function with a noise function that already resembles it leaves the network a smaller, more structured residual to learn. In function space, transport needs samples rather than densities, for without a uniform reference measure densities and many classical OT tools are unavailable, but minibatch entropic OT needs only pairwise costs, and its plan keeps the absolute-continuity structure that FFM relies on. A kernel is the way to choose the geometry, bringing prior knowledge about the data into the pairing, since bounded costs keep Sinkhorn well conditioned, Sobolev norms see derivatives, and the signature kernel sees the order and roughness of paths, while the resulting Hilbert Sinkhorn Divergence sits between kernel OT and MMD. And the change is cheap and leaves everything else alone, adding 5–7% to training at our largest resolution without touching the network, the loss or the sampler. If you generate time series, paths or PDE fields with flow matching, we hope kFFM is a useful drop-in change, and the code is on GitHub.

This work was partially supported by NSF 2303037, NSF 2312501, NSF 2531008, NSF 2553619, SRC JUMP 2.0 Center, CDSC Industry Partners, Amazon Research Awards, Snapchat, and Google Gifts. Barbora Barancikova is supported by UK Research and Innovation [UKRI Centre for Doctoral Training in AI for Healthcare grant number EP/S023283/1]. Part of this work was done while Fred Xu was an intern at Block, Inc., and we thank Block, Inc. for its support. The toy-coupling panels are cropped from a figure in the paper, and the turbulent samples and statistics are from the paper; the other figures were made for this post, with synthetic examples except for the paired-test and cost charts, which use the numbers in the paper.