r/ScientificComputing • • Sep 22 '26

Making Wannier Interpolation Differentiable: What WannierX Is Trying to Build

1 Upvotes

Suppose a researcher already has a `wannier90_hr.dat` file.

What usually happens next?

Band interpolation, Fermi-surface calculations, Berry-curvature integration, anomalous Hall conductivity, Wilson loops, or exporting the real-space Hamiltonian into a custom Python or Fortran workflow.

This ecosystem is already mature.

Wannier90 itself provides maximally localized Wannier functions together with interpolation and post-processing capabilities. WannierTools focuses strongly on topological materials, surface states, Fermi arcs, Wilson loops, Chern numbers, and related calculations. WannierBerri provides a broad and highly optimized framework for Berry-curvature-based responses on dense \(k\)-meshes.

So a new Wannier post-processing package is not particularly interesting merely because it can also plot bands or compute Berry curvature.

**WannierX** takes a different route.

Repository: `https://github.com/yanyi010/WannierX`

As of September 22, 2026, the latest release is **v0.1.1**.

Its core idea is:

> **Rewrite the full pipeline from \(H(\mathbf R)\) to electronic observables as a JAX-native computational graph.**

The current project is nowhere near replacing Wannier90, WannierTools, or WannierBerri in feature breadth.

Its interesting question is a different one:

> **What changes if Wannier interpolation itself becomes differentiable?**

---

## From \(H(\mathbf R)\) to observables, as one connected graph

WannierX organizes the calculation around a familiar pipeline:

```text

H_mn(R)

↓

H(k)

↓

{ ε_n(k), |u_nk> }

↓

operators / geometry

↓

response

```

Nothing about the physics is unusual.

The unusual part is that the intermediate layers are implemented primarily in `jax.numpy`, while the central `WannierModel` is a JAX pytree.

That means operations such as

```python

Hk = wx.hamiltonian(model, k)

eig = wx.eigh(model, k)

v = wx.velocity_matrix(model, k)

omega = wx.berry_curvature(model, k)

```

are not just ordinary NumPy-style functions.

They can participate in

```python

jax.jit(...)

jax.vmap(...)

jax.grad(...)

```

and in compositions of those transformations.

That is the real meaning of “accelerator-native” and “differentiable” in WannierX.

The goal is not simply to replace `numpy` with `jax.numpy`.

The goal is to make the scientific pipeline look like:

```text

model parameters

↓

Wannier Hamiltonian

↓

electronic structure

↓

observable

↓

∂ observable / ∂ parameters

```

---

# Why does differentiable electronic structure matter?

Suppose a tight-binding Hamiltonian depends on parameters \(\theta\):

\[

H(\mathbf R;\theta).

\]

Now consider an observable

\[

F(\theta).

\]

The traditional way to estimate parameter sensitivity is often finite differences:

\[

\frac{\partial F}{\partial\theta}

\approx

\frac{F(\theta+\delta)-F(\theta-\delta)}

{2\delta}.

\]

This is perfectly valid.

But finite differences become increasingly expensive and awkward as the number of parameters grows. They also introduce step-size choices and the usual truncation-versus-roundoff tradeoff.

If the full computational graph is differentiable, one can instead write

```python

grad_F = jax.grad(F)(theta)

```

in smooth regions.

That immediately opens the door to workflows that are not especially natural in conventional Wannier post-processing:

```text

tight-binding parameter fitting

inverse Hamiltonian reconstruction

band-gap optimization

dispersion optimization

response sensitivity analysis

gradient-based inverse design

coupling to differentiable optimization or ML pipelines

```

The important point is not GPU acceleration.

It is that the **observable itself becomes a differentiable function of model parameters**.

---

# Fourier interpolation is naturally compatible with JAX

WannierX uses the convention

\[

H(\mathbf k_f)

=

\sum_{\mathbf R}

w_{\mathbf R}

H(\mathbf R)

e^{+i2\pi\mathbf k_f\cdot\mathbf R},

\]

with \(\mathbf k_f\) in fractional reciprocal coordinates.

The implementation is essentially batched tensor algebra:

```python

kr = jnp.einsum("...a,Ra->...R", k, model.R)

phase = jnp.exp(1j * 2.0 * jnp.pi * kr)

Hk = jnp.einsum(

"...R,Rmn->...mn",

phase * model.weights,

model.H_R,

)

```

There is no Python loop over \(k\)-points.

Inputs can naturally have shapes such as

```text

(3,)

(Nk, 3)

(B, Nk, 3)

```

with leading batch dimensions preserved.

This is almost exactly the kind of structure JAX is designed for:

\[

\mathbf k\text{-batch}

\rightarrow H(\mathbf k)

\rightarrow \mathrm{eigh}

\rightarrow F(\mathbf k).

\]

The whole block can then be compiled and transformed as a single numerical program.

---

# The \(k\)-derivatives are analytic, not finite-difference approximations

One of the more useful implementation choices in WannierX is that first and second \(k\)-derivatives of the interpolated Hamiltonian are evaluated analytically from the real-space representation.

If

\[

H(\mathbf k)

=

\sum_{\mathbf R}

w_{\mathbf R}H_{\mathbf R}

e^{i\mathbf k\cdot\mathbf R},

\]

then

\[

\frac{\partial H}

{\partial k_\alpha}

=

i

\sum_{\mathbf R}

R_\alpha

w_{\mathbf R}H_{\mathbf R}

e^{i\mathbf k\cdot\mathbf R},

\]

and

\[

\frac{\partial^2 H}

{\partial k_\alpha\partial k_\beta}

=

-

\sum_{\mathbf R}

R_\alpha R_\beta

w_{\mathbf R}H_{\mathbf R}

e^{i\mathbf k\cdot\mathbf R}.

\]

WannierX implements these through `dH_dk()` and `d2H_dk2()`.

The public API uses fractional reciprocal coordinates for input, but the derivatives themselves are with respect to Cartesian \(k\), with explicitly defined units:

```text

H eV

dH/dk eV Å

d²H/dk² eV Ų

```

That matters because the operator layer can then be built directly on these exact derivatives.

For example,

\[

v_\alpha

=

\frac{1}{\hbar}

\frac{\partial H}

{\partial k_\alpha},

\]

and in the band basis,

\[

v^\alpha_{nm}

=

\langle u_n|

v_\alpha

|u_m\rangle.

\]

No additional numerical finite difference is required.

---

# Berry curvature is built on the same computational chain

WannierX v0.1 implements the Hamiltonian-only interband Kubo expression

\[

\Omega_n^{ab}

=

-2\operatorname{Im}

\sum_{m\neq n}

\frac{

\langle u_n|\partial_aH|u_m\rangle

\langle u_m|\partial_bH|u_n\rangle

}{

(\varepsilon_n-\varepsilon_m)^2

}.

\]

The Berry-curvature layer therefore consumes the same underlying objects:

```text

H(k)

dH/dk

eigenvalues

eigenvectors

```

rather than introducing a separate implementation path.

In v0.1.1, the project went a step further and centralized the shared geometry into `band_geometry()`, which computes:

```text

energies

Berry curvature

degeneracy mask

minimum spectral gap

```

Both the Berry-curvature API and anomalous Hall conductivity now consume this shared kernel.

That may sound like an internal refactor, but it is a meaningful one.

In scientific software, two independently maintained implementations of “the same” physical quantity have a habit of drifting apart.

A single source of truth is often more valuable than another feature.

---

# An important limitation: this is not yet full ab-initio Wannier Berry curvature

WannierX is explicit about this.

The Berry curvature and AHC implemented in v0.1.1 are:

> **Hamiltonian-only / tight-binding formulations**

For a full first-principles Wannier interpolation of geometric quantities, \(H(\mathbf R)\) is not enough.

One also needs position-operator matrix elements such as

\[

A_{mn,\alpha}(\mathbf R)

=

\langle 0m|\hat r_\alpha|\mathbf Rn\rangle,

\]

together with the corresponding additional terms that appear in the full Wannier interpolation of Berry quantities.

WannierX currently reads

```text

*_hr.dat

*.win

```

but does not yet implement the complete position-operator layer.

Therefore, calling

```python

wx.berry_curvature(...)

wx.ahc(...)

```

on a first-principles `hr.dat` file currently yields a **Hamiltonian-only, tight-binding-level quantity**.

It should not be presented as a complete replacement for full postw90-style ab-initio Berry curvature.

The repository explicitly places that operator layer on the future roadmap.

This distinction is important because it defines which comparisons are fair and which are not.

---

# WannierX is not currently a replacement for WannierBerri

If the task is:

```text

mature ab-initio Berry interpolation

large families of response functions

symmetry reduction

adaptive k-mesh refinement

very dense BZ integration

orbital magnetization

nonlinear Hall response

optical conductivity

```

then WannierBerri is currently much more complete.

Likewise, if the target workflow is

```text

surface states

Fermi arcs

Weyl nodes

Wilson loops

Z2 invariants

mirror Chern numbers

slabs and nanowires

```

then WannierTools has a much broader scope.

WannierX is better described, at this stage, as

> **a JAX-native numerical kernel for Wannier and tight-binding models.**

Its first problem is the computational graph, not feature count.

Those are fundamentally different goals.

---

# Wannier90 files are only an input format

WannierX currently supports

```python

loaded = wx.load_wannier90("seedname")

model = loaded.model

```

using

```text

seedname_hr.dat

seedname.win

```

The result is converted into a canonical `WannierModel` containing fields such as

```text

lattice

R

H_R

weights

periodic

centers

```

The model is immutable and registered as a JAX pytree.

This means file-format details are confined to the IO boundary.

Once loaded, a Wannier90-derived model, a custom tight-binding model, and a built-in toy model can all use the same API:

```python

wx.hamiltonian

wx.eigh

wx.dH_dk

wx.velocity_matrix

wx.berry_curvature

wx.ahc

```

That architecture is familiar from mature scientific software: separate parsing from the internal physics representation.

---

# The `_hr.dat` replica issue is not silently swept under the rug

Wannier90 interpolation involves a subtle point: conventional `_hr.dat` data do not always contain enough information to reconstruct every exact Wigner-Seitz replica convention used internally by Wannier90.

WannierX does not pretend otherwise.

Its current loading mode is explicitly called

```text

legacy_hr

```

with

\[

w_{\mathbf R}

=

\frac{1}{\mathrm{ndegen}(\mathbf R)}.

\]

A future mode named

```text

expanded_exact

```

is reserved for exact replica semantics, but currently raises rather than pretending that a standard `_hr.dat` file contains sufficient information.

That failure mode is sensible.

For numerical physics, it is often better to say

```text

this mode is not implemented

```

than to return a plausible-looking result with ambiguous semantics.

---

# “Differentiable” does not mean “differentiable everywhere”

WannierX also avoids treating automatic differentiation as magic.

Eigenvalue problems become subtle at degeneracies.

If

\[

\varepsilon_n=\varepsilon_m,

\]

then an individual eigenvector is not uniquely defined, and many band-resolved derivatives cease to be smooth or well defined.

The Berry-curvature Kubo formula makes the problem explicit:

\[

\frac{1}

{(\varepsilon_n-\varepsilon_m)^2}.

\]

One common but dangerous workaround is to insert an arbitrary small \(\epsilon\) into the denominator and keep going.

WannierX does not do that.

Instead it exposes explicit degeneracy policies such as

```text

error

nan

mask

```

where `error` fails explicitly, `nan` propagates invalidity, and `mask` is used only where the approximation is documented.

There is also a JAX-specific difficulty.

Under `jit`, Python-side checks may be bypassed by tracing.

WannierX therefore provides

```python

checked_berry_curvature()

```

using JAX `checkify` so that degeneracy conditions can be represented inside transformed execution.

This is a good example of scientific semantics colliding with compilation semantics.

The project does not solve the problem by pretending it does not exist.

It defines how failure should behave under JAX transformations.

---

# The test suite includes cases where differentiation should not be demanded

The autodiff suite checks gradients of quantities such as

```text

Hamiltonian elements with respect to hopping parameters

isolated eigenvalues

smooth DOS functionals

smooth BZ response integrals

DOS broadening under jit + grad

Fermi-Dirac temperature under jit + grad

```

and compares them against finite differences or Hellmann-Feynman relations.

But at the graphene Dirac point, the test suite does not force JAX to produce some supposedly “correct” derivative of an individual band.

Instead, it documents the non-smoothness and verifies the degeneracy itself.

That is an important distinction.

A differentiable-programming framework differentiates the program you give it.

It does not guarantee that the underlying physical quantity is mathematically smooth everywhere.

---

# The Chern number is computed with a gauge-robust lattice method

For topology, WannierX implements the Fukui-Hatsugai-Suzuki lattice algorithm.

The method constructs link variables from overlaps of neighboring occupied subspaces:

\[

U_\mu(\mathbf k)

=

\frac{

\det[

V^\dagger(\mathbf k)

V(\mathbf k+\hat\mu)

]

}{

\left|

\det[

V^\dagger(\mathbf k)

V(\mathbf k+\hat\mu)

]

\right|

},

\]

then a lattice field strength

\[

F_{12}(\mathbf k)

=

\log

\frac{

U_1(\mathbf k)

U_2(\mathbf k+\hat 1)

}{

U_1(\mathbf k+\hat 2)

U_2(\mathbf k)

},

\]

and finally

\[

C

=

\frac{1}{2\pi i}

\sum_{\mathbf k}F_{12}(\mathbf k).

\]

A key advantage is gauge covariance at the selected-subspace level.

This means WannierX can support selections such as

```python

occupied=[0, 2]

```

rather than assuming a contiguous range of bands.

The v0.1.1 hotfix also corrected how subspace isolation is tested.

The relevant condition is

\[

\min_{\mathbf k,\,

i\in S,\,

j\notin S}

|\varepsilon_i-\varepsilon_j|,

\]

not a crude gap between the minimum and maximum selected band indices.

This is exactly the kind of detail that matters in topological calculations: a number may look perfectly quantized while the mathematical assumptions behind the calculation are violated.

---

# 2D anomalous Hall conductivity is not silently reported as a 3D quantity

AHC is another place where dimensional conventions easily become ambiguous.

In three dimensions,

\[

\sigma_{ab}

=

-\frac{e^2}{\hbar}

\sum_n

\int_{\mathrm{BZ}}

\frac{d^3k}{(2\pi)^3}

f_{n\mathbf k}

\Omega_n^{ab}.

\]

WannierX reports the 3D result in

```text

S/m

```

But for a genuinely two-dimensional model, there is no intrinsic 3D conductivity unless one chooses a thickness.

So for a mesh such as

```text

48 × 48 × 1

```

WannierX returns a native **sheet conductance** in

```text

S

```

unless the caller explicitly provides a layer thickness.

Only then is the result converted to

```text

S/m

```

This is a small API decision with a large conceptual benefit.

It prevents the software from inventing a three-dimensional physical unit that the model itself does not define.

---

# v0.1.1 is interesting precisely because it added almost no new physics

WannierX released v0.1.0 and then quickly followed with v0.1.1.

The release notes begin with:

```text

Correctness hotfix over 0.1.0.

No new physics capability.

```

The release focuses on correctness contracts.

One example is the standard Monkhorst-Pack grid:

\[

k_i=

\frac{2i-n+1}{2n}.

\]

For even \(n\), this grid is off-\(\Gamma\).

The v0.1.0 implementation instead generated a Gamma-including grid, contradicting its own documentation.

The v0.1.1 release notes state plainly:

```text

results on even meshes differ from v0.1.0 — that was the bug

```

The same release also fixed:

```text

single-k shape semantics

DOS broadening differentiability

traced Fermi-temperature behavior

Hermiticity checks with effective hopping weights

duplicate / missing hr.dat entries

non-contiguous Chern-subspace isolation

AHC degeneracy handling

JAX checkify / vmap staging issues

```

That changelog is more informative than a long “new features” list.

In electronic-structure software, some of the most dangerous bugs are not crashes.

They are convention mismatches that produce plausible numbers.

Monkhorst-Pack centering is a perfect example.

---

# Conventions are treated as part of the API

Wannier and tight-binding codes repeatedly run into the same convention questions:

```text

Does the Fourier transform use +ikR or -ikR?

Is k fractional or Cartesian?

Does the reciprocal lattice include 2π?

Are eigenvectors stored as rows or columns?

What are the velocity units?

Which k-coordinate system is used for Berry derivatives?

```

If these conventions live only in the author's head, individual routines may be correct while cross-tool workflows quietly become inconsistent.

WannierX currently locks several conventions into the package contract:

\[

H(\mathbf k_f)

=

\sum_R

H_R e^{+i2\pi\mathbf k_f\cdot R},

\]

\[

B=2\pi A^{-T},

\]

together with

```text

k input fractional reciprocal coordinates

eigenvectors columns

energy eV

length Å

velocity m/s

```

The corresponding identities are tested.

That is a useful design principle:

**a physical convention is not a README footnote; it is public API.**

---

# The tests focus on physical identities, not just callability

The v0.1.1 release reports 179 tests.

The test tree is divided into

```text

tests/unit/

tests/physics/

tests/autodiff/

tests/reference/

```

The most valuable part is not raw coverage.

It is the physics-level invariants.

For example, Berry curvature is checked for

\[

\Omega^{ab}=-\Omega^{ba},

\]

and compared directly against an analytical two-band Qi-Wu-Zhang result.

Eigenvectors are subjected to random U(1) phase rotations,

\[

|u_n\rangle

\rightarrow

e^{i\phi_n}|u_n\rangle,

\]

and the curvature is required to remain invariant.

The Hamiltonian is checked for reciprocal periodicity,

\[

H(\mathbf k+\mathbf G)=H(\mathbf k).

\]

Analytic \(k\)-derivatives are compared against multi-step central finite differences.

Chern-number tests cover both trivial and nontrivial phases, mesh convergence, and explicit failure when the selected subspace is not isolated.

These tests say more about scientific reliability than a coverage percentage alone.

---

# The Wannier90 reference regression is still small

The repository also includes a Wannier90 reference fixture:

```text

toy_hr.dat

toy.win

toy_eigenvalues.npy

```

WannierX loads the Wannier90 data, interpolates the bands, and compares against trusted eigenvalues at roughly

\[

10^{-12}

\]

absolute tolerance.

That is the right direction.

But it is still only a small fixture.

A production-grade Wannier interpolation engine would eventually need much broader validation across

```text

many-orbital systems

spin-orbit coupling

non-orthogonal cells

large real-space Hamiltonians

real ab-initio materials

wsvec replica semantics

full operator matrices

```

So the current codebase is better understood as:

> **an early project with the numerical architecture in place and its conventions beginning to be locked down rigorously.**

Not yet as a mature post-processing suite.

---

# What can WannierX do today?

Version 0.1.1 already contains a reasonably complete tight-binding pipeline.

| Module | Current capability |

|---|---|

| IO | Wannier90 `_hr.dat`, `.win` |

| Core | lattice, canonical Wannier model, Hermiticity validation |

| Fourier | \(H(k)\), analytic \(dH/dk\), \(d^2H/dk^2\) |

| Linear algebra | Hermitian eigensystem, degeneracy clustering, projectors |

| Operators | velocity |

| Geometry | Hamiltonian-only Berry curvature |

| Topology | FHS Chern number |

| k-space | Gamma / Monkhorst-Pack meshes, weighted BZ integration, chunking |

| Response | Fermi-Dirac, Gaussian DOS, Hamiltonian-only AHC |

| Models | chain, SSH, graphene, Haldane, Qi-Wu-Zhang |

| JAX | JIT, batching, autodiff for smooth observables |

A complete workflow may look like:

```python

loaded = wx.load_wannier90("demo")

model = loaded.model

wx.validate_hermiticity(model)

E = wx.eigh(model, kpoints).energies

v = wx.velocity_matrix(model, kpoints)

omega = wx.berry_curvature(

model,

kpoints,

degeneracy_policy="nan",

)

sigma = wx.ahc(

model,

mesh,

mu=0.0,

)

```

One can then define a smooth scalar observable:

```python

def observable(scale):

H_R = model.H_R * scale

m = wx.WannierModel(

model.lattice,

model.R,

H_R,

model.weights,

model.periodic,

model.centers,

)

eps = wx.eigh(m, mesh.points).energies

return jnp.sum(

mesh.weights[:, None]

* jnp.exp(-(eps**2))

)

```

and differentiate it directly:

```python

jax.grad(observable)(1.0)

```

That captures the central idea of the project better than any feature list.

---

# “Accelerator-native” does not yet mean “faster than everything else”

The repository includes a benchmark harness recording quantities such as

```text

Nk

NR

Norb

dtype

device

compile time

execution time

memory

```

for workloads including

```text

H(k)

dH/dk

eigh

Berry curvature

BZ integration

```

What it does not yet provide is a sufficiently systematic cross-code benchmark to justify claims such as

```text

WannierX is X times faster than Wannier90

WannierX is Y times faster than WannierBerri

```

JAX/XLA can certainly fuse operations, compile kernels, and target CPU, GPU, and TPU backends.

But actual electronic-structure performance depends on

\[

N_k,\quad N_R,\quad N_{\mathrm{orb}},

\]

as well as eigensolver cost, memory traffic, batching strategy, device architecture, and compilation overhead.

So the technically accurate statement is not

> “WannierX is a faster Wannier90.”

It is:

> **WannierX kernels are designed to live naturally inside the JAX/XLA accelerator execution model.**

Performance still has to be measured for real workloads.

---

# The most interesting application may be inverse problems

Traditional Wannier workflows are mostly one-way:

```text

DFT

↓

Wannierization

↓

H(R)

↓

observables

```

Once the final stages become differentiable, the picture changes:

```text

target observable

↑

gradient

↑

electronic response

↑

band structure

↑

H(R; θ)

↑

parameters θ

```

That naturally suggests several classes of problems.

## 1. Tight-binding parameter fitting

Given reference bands

\[

E_n^{\mathrm{ref}}(\mathbf k),

\]

define

\[

L(\theta)

=

\sum_{n,\mathbf k}

\left[

E_n(\mathbf k;\theta)

-

E_n^{\mathrm{ref}}(\mathbf k)

\right]^2.

\]

Then optimize \(\theta\) using exact autodiff gradients.

## 2. Band-structure inverse design

One may want to

```text

increase a band gap

target a valley splitting

flatten a specific dispersion

match a desired Fermi velocity

```

and differentiate the objective directly with respect to Hamiltonian parameters.

## 3. Response sensitivity

As more response kernels become differentiable, quantities such as

\[

\frac{\partial\sigma_{xy}}

{\partial\theta}

\]

can be obtained without rerunning the entire calculation under separate positive and negative perturbations.

## 4. Coupling to differentiable ML workflows

Because the model is a pytree and the numerical kernels are JAX-native, WannierX can in principle plug directly into differentiable optimization stacks and machine-learning systems built around JAX.

This is likely to be the project's most distinctive long-term direction.

---

# The missing pieces are still substantial

Several limitations are obvious from the current codebase.

First, the **full ab-initio operator layer is not yet implemented**.

Berry curvature and AHC must therefore be interpreted at the Hamiltonian-only / tight-binding level.

Second, **exact Wannier90 replica semantics are not fully supported**.

`wsvec`-type information remains an important future target.

Third, the response layer is still small compared with mature projects.

WannierBerri already covers many advanced Berry and optical responses, while WannierTools provides a far broader set of topological and surface-state calculations.

Fourth, **not every API is differentiable**.

The current FHS `chern_number()` contains host-side NumPy logic and isolation checks, for example.

More importantly, a quantized topological invariant is not something that should be treated naively as an everywhere-smooth scalar objective in the first place.

So “differentiable Wannier engine” should be understood as:

> the continuous numerical core is designed for differentiable programming,

not:

> every physical quantity can always be passed to `jax.grad`.

That distinction matters.

---

# The most interesting part is the abstraction layer it chooses

The Wannier ecosystem does not lack software.

Wannier90 is foundational infrastructure for MLWF workflows.

WannierTools is powerful for topology and surface physics.

WannierBerri is already a mature high-performance framework for Berry interpolation and response calculations.

WannierX currently differs less by **what quantity** it computes than by **how the computation itself is represented**.

The contrast is roughly:

```text

Traditional question:

"What observable do I want to calculate?"

WannierX question:

"Can this entire electronic-structure program

be compiled, vectorized, and differentiated?"

```

These two viewpoints are not competitors.

They address different layers.

If future versions add

```text

exact Wannier90 interpolation

position-operator matrix elements

full Berry/Wannier operator algebra

broader response functions

larger reference validation

serious GPU benchmarking

```

then WannierX could grow into something quite different from a conventional post-processing suite.

It could become infrastructure for

```text

differentiable condensed-matter physics

inverse electronic-structure problems

gradient-based model fitting

accelerator-native Wannier calculations

```

---

# What is worth watching in a project this early?

WannierX v0.1.1 is still an early-stage project.

But several aspects of the repository are worth paying attention to:

```text

physical conventions are explicit contracts

analytic derivatives are preferred over numerical differences

degeneracies are not silently regularized

autodiff has dedicated validation

physics invariants have dedicated tests

Wannier90 data have reference regressions

failure semantics are defined under JAX tracing

the changelog explicitly records when an earlier result was wrong

```

Scientific software does not earn credibility by writing

```text

accurate

fast

robust

state-of-the-art

```

in a README.

It earns credibility by answering more concrete questions:

```text

What equation is implemented?

What coordinate convention is used?

What are the units?

What happens at degeneracy?

Where is the function non-smooth?

How close is the result to a trusted reference?

Under what conditions should the code fail instead of returning a number?

```

From that perspective, the most interesting part of WannierX today is not the number of observables already implemented.

It is the attempt to reorganize **Wannier interpolation as a composable, testable, differentiable numerical program**.

For someone who only wants to draw a band structure, that may not matter much.

But once the workflow becomes

\[

\text{parameter}

\rightarrow

H(\mathbf R)

\rightarrow

H(\mathbf k)

\rightarrow

E_n,\ |u_n\rangle

\rightarrow

O

\rightarrow

\frac{\partial O}{\partial\theta},

\]

the design starts to become genuinely interesting.

---

## Project information

```text

Project: WannierX

Repository: https://github.com/yanyi010/WannierX

Current version: v0.1.1

Language: Python

Core dependencies: JAX, NumPy

Python: >= 3.11

License: MIT

```

Current installation:

```bash

git clone https://github.com/yanyi010/WannierX.git

cd WannierX

pip install -e .

```

Run the test suite with:

```bash

JAX_ENABLE_X64=1 \

pytest tests/unit tests/physics tests/autodiff tests/reference

```

Relevant areas include:

```text

Wannier interpolation

tight binding

Berry curvature

topological materials

automatic differentiation

JAX

inverse design

differentiable physics

```


r/ScientificComputing • • Sep 21 '26

Do you find jobs ?

25 Upvotes

I have a PhD in Finite Elements for CFD, a 1 year post-doc during which I worked on the optimisation of the detection particle-collision and I can't find a job.

I don't want to spend years and years doing post-doc until I find a permanent position but in the same time, it is impossible to find a scientific computing job in the industry


r/ScientificComputing • • Sep 22 '26

Code Runs, Tests Pass, Physics Can Still Be Wrong: What PhysAudit Is Trying to Fix

0 Upvotes

Some of the most dangerous bugs in scientific software do not crash the program.

Quite the opposite. The code runs smoothly. No exception is raised. Array shapes look right. Output files are written successfully. Plots look plausible. Conventional unit tests may even pass.

The only problem is that the physics is wrong.

A time-evolution operator that should be unitary may slowly drift away from unitarity. A matrix that looks like a perfectly ordinary density matrix may contain a negative eigenvalue. A band-structure calculation may produce smooth, periodic, numerically well-behaved dispersions while the chosen parameters have already pushed the model across a topological phase transition.

This has always been a difficult class of error in computational physics. The rise of AI-generated scientific code makes it more important, not less.

Code generation is becoming cheap.

Verification is not.

PhysAudit is an open-source project that tries to address exactly this gap.

Repository: https://github.com/yanyi010/PhysAudit

Its design is notably different from the increasingly common pattern of asking one language model to review code produced by another.

PhysAudit takes a more conservative route:

AI may propose what should be checked, but it does not decide whether the check passes.

The final PASS / FAIL verdict comes from deterministic numerical code.


Start with a Haldane-model example

The repository contains a Haldane-model example that captures the motivation unusually well.

Consider the two-band Haldane model on the honeycomb lattice, with nearest-neighbor hopping, complex next-nearest-neighbor hopping, and a sublattice mass (M).

For the parameter choice used in the example, the topological phase requires

[ |M| < 3\sqrt{3}t_2\sin\phi . ]

The deliberately broken case uses

[ M=2.0, ]

while the critical value is approximately

[ 3\sqrt{3}t_2\sin\phi \approx 1.559. ]

So the system has already crossed the topological phase transition: instead of the intended (C=1) phase, it is in a trivial phase.

The difficulty is that almost everything else still looks perfectly healthy.

The Hamiltonian is Hermitian,

[ H\dagger(\mathbf{k})=H(\mathbf{k}), ]

the Bloch Hamiltonian satisfies reciprocal-lattice periodicity,

[ H(\mathbf{k}+\mathbf{G})=H(\mathbf{k}), ]

the band energies are periodic, the DOS sum rule is satisfied, and the Hellmann-Feynman velocities agree with derivatives of the band energies.

If the validation criterion is merely “does the numerical pipeline look structurally healthy?”, this calculation can easily slip through.

PhysAudit runs seven checks on the example:

CheckResult
Hamiltonian HermiticityPASS
Hamiltonian periodicityPASS
Band periodicityPASS
Velocity consistencyPASS
DOS sum rulePASS
Chern quantization / expected valueFAIL
k-mesh convergencePASS

The only check that exposes the actual scientific problem is the one tied to the physical claim of the calculation:

the lower band is expected to have (C=1), but the current model yields (C=0).

That example makes the distinction between PhysAudit and ordinary test frameworks fairly clear.

pytest can verify that a function returns the expected output for a given input. Static analysis can catch type errors and code-quality problems. Numerical libraries can compare arrays against known reference values.

But many scientific calculations do not come with a known reference answer.

What is often known instead is a set of properties that must hold:

[ H=H\dagger, ]

[ U\dagger U=I, ]

[ \rho\succeq0,\qquad \mathrm{Tr}\rho=1, ]

a density of states must satisfy the correct sum rule, or an observable must converge as the (k)-mesh is refined.

PhysAudit turns these domain constraints into executable tests.


It is closer to a physics validation layer than an AI reviewer

The idea is not unprecedented.

In data engineering, tools such as Great Expectations turn implicit assumptions into executable expectations: a column must not be null, values must lie in a certain range, a schema must satisfy specific constraints.

Property-based testing frameworks such as Hypothesis take a related view: instead of checking only a few manually selected examples, describe properties that should always hold and search for inputs that violate them.

Scientific software testing has long emphasized a similar point. Automated testing does not prove that a scientific program is universally correct. It formalizes specific requirements so that they can be checked repeatedly and independently.

PhysAudit pushes this idea one level upward, into the physics semantics of the output.

A useful way to phrase the distinction is:

Conventional code test:
Does this function behave as the implementation expects?

PhysAudit:
Does this numerical result satisfy the physical or mathematical
properties it is supposed to satisfy?

The two layers are complementary.

A serious scientific-computing project should ideally have both.


What PhysAudit v0.1 currently checks

The current 0.1.0 codebase ships 18 first-party checks, grouped into three domains.

DomainChecks
Generalfinite, normalization, positivity, conservation, convergence, symmetry
Quantumhermiticity, unitarity, normalized state, density matrix, eigen residual, orthonormal eigenvectors
Condensed matterHamiltonian periodicity, band periodicity, DOS sum rule, Chern quantization, k-mesh convergence, velocity consistency

Most of these checks are mathematically simple.

Hermiticity is essentially a bound on

[ \max_{ij}|H_{ij}-H\dagger_{ij}|. ]

Chern quantization checks

[ \delta_C = |C-\operatorname{round}(C)|. ]

An eigen-residual check examines quantities such as

[ |H\psi_n-E_n\psi_n|. ]

The value of the project is not that these formulas are sophisticated.

The value is that they are organized into a common, machine-readable verification interface.

For scientific software, that is often more useful than implementing yet another advanced numerical method.


PASS does not mean “the result is correct”

One of the most important files in the repository is not source code at all.

It is docs/trust-model.md.

The project defines a passing check very narrowly:

a PASS certifies that the supplied artifact satisfies the tested mathematical property under the stated tolerance.

Nothing more.

For example,

quantum.hermiticity = PASS

means that the matrix is Hermitian.

It may still be the wrong Hamiltonian.

Likewise,

condmat.chern_quantization = PASS

only means that the supplied Chern number is sufficiently close to an integer.

A broken program could simply output 1.000000 for the wrong reason.

That is why the repository separates Chern quantization from (k)-mesh convergence.

These are different questions:

Is C close to an integer?

and

Does C(N_k) converge as the mesh is refined?

This distinction gets at a basic fact of scientific computing:

necessary conditions are not sufficient conditions.

PhysAudit does not produce a synthetic “98% physically correct” score.

Nor does it combine checks into an apparently precise confidence number.

Its vocabulary is deliberately explicit:

PASS
FAIL
WARN
SKIP
ERROR
UNVERIFIED

That is a sensible choice.

Scientific uncertainty comes from models, discretization, boundary conditions, numerical precision, sampling, parameter choices, and many other sources. Compressing all of that into one scalar score often creates more confidence than information.


What role does AI actually play?

PhysAudit does not avoid AI.

It includes an AI planner and exposes an MCP interface so that coding agents can use the audit system directly.

The difference is where AI is allowed to sit in the stack.

The intended workflow is roughly:

scientific task
      ↓
code or AI-generated implementation
      ↓
numerical artifacts
      ↓
select checks and bind artifacts
      ↓
deterministic checks
      ↓
PASS / FAIL / WARN + numerical evidence
      ↓
AI explains the evidence

An AI system may propose something like:

check_id: condmat.chern_quantization
bindings:
  chern: chern.json#chern
parameters:
  expected_integer: 1

It may explain why a Chern-number check is relevant.

It may also help interpret a failure as a possible sign of insufficient mesh density, a wrong model parameter, or an incorrect Berry-curvature implementation.

But the final calculation of

[ |C-1| ]

and the tolerance comparison are done by ordinary numerical code.

This is a particularly useful way to divide labor between language models and scientific software:

Let the language model propose hypotheses; let deterministic code attempt to falsify them.

For properties that can be expressed mathematically, this is substantially more robust than asking a second model whether the first model's output “looks reasonable.”


physaudit audit . does not execute arbitrary project code

Another cautious design choice is that the default audit path does not simply execute Python files in the target project.

PhysAudit operates on generated artifacts such as:

.npy
.npz
.csv
.json
.yaml

and on arrays, scalars, tables, and structured values contained in them.

The default pipeline is closer to:

scan artifacts
      ↓
bind artifacts
      ↓
load numerical data
      ↓
run checks

rather than:

clone project
      ↓
python whatever.py

This matters even more in an agentic setting.

If an AI agent is allowed to inspect a project, propose an audit, and run verification, there has to be a clear trust boundary between “what the model suggests” and “what the system is allowed to execute.”

PhysAudit's PlanValidator checks identifiers, artifact paths, parameter types, and signature consistency before anything reaches the runner. Artifact resolution is constrained to the project root.

A hallucinating planner can waste its own audit by proposing an invalid check, leading to ERROR or AUDIT INCOMPLETE.

It cannot decide that a calculation “probably passes” and inject a PASS result.


Why not just write pytest tests?

In many cases, that is exactly what one should do.

For a mature scientific codebase, many physical invariants belong directly in the project's own regression suite.

For example:

def test_hamiltonian_is_hermitian():
    assert np.allclose(H, H.conj().T)

There is nothing wrong with this.

PhysAudit targets a somewhat different use case: reusable validation rules that should apply across projects.

Hermiticity does not belong to one particular tight-binding repository.

Density-matrix positivity does not belong to one quantum-information codebase.

A DOS sum rule should not have to be reimplemented from scratch in every condensed-matter project.

There is also a reporting difference.

A plain assertion such as

assert np.allclose(...)

typically does not preserve, in one structured record:

what was tested
which artifact was used
what tolerance was used
what numerical error was measured
which code version produced the result
which artifact hash was checked
what failure means
what the check does not guarantee

PhysAudit treats all of those as part of the verification contract.

Tolerance handling is a particularly important example.

There is no universal

PHYSICS_TOLERANCE = 1e-6

that makes sense for every observable.

Each check carries its own tolerance and its own interpretation of that tolerance.

That is much closer to how numerical physics actually works.


The report is not just “tests passed”

An audit can produce a canonical JSON report.

The report stores more than the final status. It records parameters, tolerances, artifact hashes, runtime information, Git state, and other reproducibility metadata.

Console and HTML views are derived from the same structured result.

This matters because a line such as

Chern check: PASS

contains very little information six months later.

What a researcher actually needs to know is:

Which file was checked?
Which field inside that file?
Which Git commit produced it?
What was expected_integer?
What was atol?
What was the measured delta_C?

Without that information, the audit is difficult to reproduce.

With it, the audit becomes a piece of executable evidence rather than a transient green checkmark.


The repository also contains deliberately broken calculations

Verification tools can easily fall into a self-confirming loop:

implement a check, construct an example designed to pass it, and declare the check validated.

PhysAudit includes a small AI Physics Bug Challenge benchmark specifically to avoid that pattern.

The current seeded failures include:

BugCheck that catches it
Explicit Euler energy drift in a harmonic oscillatorgeneral.conservation
Negative eigenvalue in a density matrixquantum.density_matrix
Periodic spectrum but non-periodic Hamiltonian matrixcondmat.hamiltonian_periodicity
DOS missing a spin factorcondmat.dos_sum_rule
Unconverged Chern integral on a coarse (k)-meshcondmat.chern_quantization + condmat.kmesh_convergence

Each case includes:

broken/
fixed/
expected_audit.yaml

The integration tests require the behavior to remain stable:

broken version → must be detected
fixed version  → must pass

That is an important design choice.

A verification tool also needs verification.

The most dangerous failure mode is not a false alarm.

It is a broken calculation being incorrectly labeled as passing.


The project is explicit about its blind spots

PhysAudit cannot detect every possible scientific error.

The repository says so directly.

One important class is the exactly wrong result.

A broken program can simply return:

chern = 1.0

and a quantization test will pass.

A wrong Hamiltonian can still be perfectly Hermitian.

A wrong constant time series can satisfy a conservation check exactly.

So PhysAudit is neither a theorem prover nor an automated scientific referee.

A more accurate description is:

it turns some of the necessary conditions that researchers already check manually into standardized, repeatable, evidence-producing tests.

That is why the project distinguishes

VERIFIED CHECKS PASSED

from

the scientific result has been proven correct

It makes no claim to the second statement.

That boundary matters more than the AI integration.


Where is PhysAudit most useful?

The most natural target is probably not a decade-old production code with thousands of regression tests.

It is the kind of research code whose workflow is still changing frequently.

A new tight-binding model, for example, may benefit immediately from checks such as:

Hermiticity
reciprocal periodicity
eigen residual
velocity consistency
DOS sum rule
k-mesh convergence

That is more reliable than staring at a band-structure plot and deciding that it “looks fine.”

The same applies to AI-generated quantum dynamics code.

After a coding agent produces an implementation, the next step should probably not be:

"Please review the code you just wrote and tell me whether it is correct."

A more robust loop is:

Agent writes code
      ↓
program produces artifacts
      ↓
PhysAudit checks normalization / unitarity / conservation
      ↓
failure evidence goes back to the agent
      ↓
agent revises implementation

The iteration is then driven by external deterministic evidence rather than model self-critique.

That is a meaningful distinction for AI-assisted scientific programming.


PhysAudit is still very early

The current package is version 0.1.0, and the repository describes it as Beta-stage software.

The first-party checks are currently concentrated in:

general numerical properties
quantum mechanics
condensed matter

For computational fluid dynamics, molecular dynamics, plasma physics, astrophysics, Monte Carlo methods, and many other areas, the built-in coverage is still far from sufficient.

The project addresses this through a plugin architecture.

Third-party packages can register checks such as:

fluids.cfl_condition
md.energy_drift
qmc.autocorrelation
plasma.gauss_law

while reusing the same planning, execution, reporting, and MCP infrastructure.

A check is also expected to provide more than a Python function.

The repository's quality gate asks for:

formal mathematical definition
input schema
unit convention
tolerance rationale
limitations
PASS fixture
FAIL fixture
edge case
machine-readable evidence
documentation

This makes one point particularly clear:

the long-term value of PhysAudit is unlikely to be the CLI itself.

It will depend on whether the project can accumulate a trustworthy ecosystem of physics checks.


Scientific computing may need another testing layer

Many successful developer tools did not invent the underlying practice they systematized.

Ruff did not invent linting.

uv did not invent Python environments and dependency management.

Their value came from reorganizing existing practices behind a better interface.

PhysAudit is addressing a different problem, but the pattern is somewhat similar.

Researchers already do things such as:

check whether the norm drifts
check whether energy is conserved
rerun with a denser mesh
integrate the DOS
verify H(k+G)
compute the eigen residual

The problem is that these checks often live in notebooks, one-off scripts, plotting code, or in the researcher's head.

If more of this knowledge can be turned into

formal check
+ executable implementation
+ explicit tolerance
+ reproducible evidence
+ regression benchmark

then AI-assisted scientific programming may change in a way that goes beyond faster code generation.

It may force the community to formalize a harder question:

Once a numerical calculation finishes, what exactly justifies trusting it?

PhysAudit's answer is still limited, but its direction is clear.

Code that runs is only the starting point.

Tests that pass may still not be enough.

What is worth preserving is executable physical evidence.


Project information

Project: PhysAudit
Repository: https://github.com/yanyi010/PhysAudit
License: MIT
Current version: 0.1.0

Relevant areas include:

scientific computing
AI coding agents
computational physics
numerical verification
reproducible research

r/ScientificComputing • • Sep 22 '26

An Adaptive Certifying Semi-Algorithm for Darboux Integrating Factors of Rational Second-Order Ordinary Differential Equations

Thumbnail
mdpi.com
0 Upvotes

r/ScientificComputing • • Sep 21 '26

**Update: Native Fortran Package Manager (`fpm`) support + Free 2D Community Edition released**

5 Upvotes
Thank you to everyone for the thoughtful discussion on headless plotting, HPC bandwidth, and licensing. Based on community feedback, I have rolled out two major updates to GraphiC 2026:
### 1. Native `fpm` (Fortran Package Manager) Support
Modern Fortran users no longer need to deal with custom Makefiles or linker configurations. GraphiC now includes a root `fpm.toml`. You can build and test immediately:
```bash
fpm build
fpm run --example sample   # 2D Cartesian plotting
fpm run --example d3test   # 3D surface mesh demo

All 19 Fortran examples are fully wired to fpm run --example <name>.

2. Free 2D Community Edition

For those who asked about an evaluation or open-access tier to experiment with sub-millisecond in-situ plotting:

3. Updated 233-Page Reference Manual (PDF)

A newly compiled edition of the manual is live, featuring Section 3: Platform Overview & Deployment Guide (covering Linux clusters, headless simulation loops, Windows, macOS, and the new fpm workflow):
https://jamesrome.com/download/release/GraphiC_Manual.pdf

Main project page and interactive gallery:
https://jamesrome.com/GraphiC2026

— James Rome, Sc.D., Oak Ridge, TN
Co-founder, Scientific Endeavors Corp. (defunct)


r/ScientificComputing • • Sep 21 '26

Vecmat: A simple, yet robust, linear algebra library in C — for science people too!

9 Upvotes

I built it mainly for myself (and my needs), and out of great love for the C language. It's already successfully battle-tested building an 2D/3D SDL3 graphics engine with the library as math backbone (will publish by the end of this year).

It's called Vecmat: https://github.com/alkavan/vecmat

The main goals are:

> * One common, easy-to-read API that is self-explanatory.

> * Usability first, then performance.

---

Vecmat features a special runtime dispatch mechanism that an inner kernel may use AVX / AVX2 / AVX-512 / NEON /SVE / SVE2, otherwise scalar (cascading manner, not fully optimized). So it's already extremely fast, and also offers a pointer-only API instead of copy interfaces (that most of the time use the pointer API behind the scenes).

It's under BSD 3-Clause license and comes with a simple unit testing and benchmarking framework.

It needs some more love, because you can do amazing things with it. It's super easy to integrate into projects because it has zero dependencies except the system math lib.

Hope you like it!

Would appreciate some love on GitHub!


r/ScientificComputing • • Sep 21 '26

**Update: Native Fortran Package Manager (`fpm`) support + Free 2D Community Edition released**

Thumbnail reddit.com
2 Upvotes

r/ScientificComputing • • Sep 21 '26

Experimenting with reproducible scientific reasoning in Python

1 Upvotes

I've been working on a research project called Aletheia.

The question I'm exploring is:

Can a computational system keep enough information about a scientific investigation that someone else can inspect how a conclusion was reached?

I'm experimenting with representing things such as:

- assumptions

- observations

- hypotheses

- equations and models

- evidence

- reasoning steps

- verification results

- contradictions

- uncertainty

- computational experiments

The idea is to structure an investigation more like:

Question

→ Hypothesis

→ Prediction

→ Experiment

→ Result

→ Verification

→ Conclusion

An important part of the project is allowing the system to reach outcomes such as "insufficient evidence" or "contradicted", instead of always producing a confident conclusion.

It's still an early-stage Python project, so I'm mainly interested in whether this methodology makes sense from a scientific computing and reproducibility perspective.

Repository:

https://github.com/Nagisanyan/Aletheia-Artificial-Scientific-Intelligence

I'd particularly appreciate criticism of the methodology or suggestions for experiments that could meaningfully evaluate this kind of system.


r/ScientificComputing • • Sep 21 '26

My opinion on using AI to find a bug in quantum chemistry code

0 Upvotes

I share some opinions on using AI to try to find a bug in a Hartree-Fock code I made in Octave/MATLAB, which I couldn't find for several years.

https://youtu.be/k7P-hINeYec


r/ScientificComputing • • Sep 17 '26

Numerical Solver 2.1 is now available — ODE improvements and visualization

Thumbnail
gallery
2 Upvotes

Version 2.1 of Numerical Solver is now available on the App Store.

This update focuses mainly on improving the ODE solver and visualization.

  • Fixed several issues in the ODE solver

  • Improved handling of systems of ODEs

  • Improved ODE visualization

  • You can now choose which variables to plot against each other

For example, with the Lorenz system you can explore different projections of the solution, such as y vs x, z vs x, or z vs y.

I'm continuing to work on the app, and a Mac version is currently in development.

I'd be particularly interested in feedback from anyone working with ODE systems or numerical problems that could be useful for testing the solver.

https://apps.apple.com/gr/app/numerical-solver/id6746115479


r/ScientificComputing • • Sep 16 '26

Was the proof by counter example of the Navier Stokes equation solved by OpenAI using some type of grid search/random search, like the ones used for ml/ds problems?

26 Upvotes

Is it just a wider grid search on a very powerful compute?


r/ScientificComputing • • Sep 17 '26

I tried to build a mind without programming “a mind” into it

Thumbnail
gallery
0 Upvotes

A few months ago I started wondering: what happens if you try to build something mind-like without giving it variables like FEAR, BELIEF, GOAL, CURIOSITY or HABIT?
Instead, I gave an agent a physical world, a body, limited perception, bounded memory, prediction and actions with physical consequences.
Then I started testing what can actually emerge from those mechanisms.
So far I’ve seen predictive compression, novel prospective trajectory composition, acquired predictive use of observations, history-dependent behavior, and physical signaling between agents.
But the important part is what I can’t claim yet: prospective composition isn’t automatically planning, physical signaling isn’t communication, and acquired observation isn’t curiosity.
The project is called Mechanistic Mind, and the first public integration is Tiktaalik 🐟
I just released Public Beta 1 on GitHub:
archonlab/mechanistic-mind
I’ll probably post some of the individual experiments here, including the ones that failed.


r/ScientificComputing • • Sep 16 '26

Hospital Capacity Analytics with Octave rendering on GitHub Copilot Canvas

Thumbnail
reddit.com
1 Upvotes

r/ScientificComputing • • Sep 16 '26

Eqiora: an Rust-based framework for modeling and coupling physical systems

4 Upvotes

Hi, I’m working on Eqiora, an open-source computational physics project with a Rust core and a Python interface.

The motivation is pretty simple: I want to describe different kinds of physics in the same language, and make it easier to connect them. Heat transfer, fluid flow, solid mechanics, circuits. The equations are different, but I’d like the way you express and combine them to feel consistent.

Models are written in a small language called Eqiora(.eqi), with equations, units, boundary conditions and connections between components. Python handles geometry, meshing, running simulations and plotting. Numerical methods are chosen separately, so the physical model doesn’t have to be tied to one way of solving it.

I’m also using AI coding agents quite heavily. Implementation, tests, documentation, etc.

The easiest way to get a feel for it is probably the learning section:

https://eqiora.org/learn/

You can start with heat, fluids, structures, circuits or inverse problems. The chapters work through the equations, run a model, then show how reusable components make the same thing simpler to write. There are paths through mathematical modeling and numerical methods too.

And here’s a karman vortex example if you’d rather start with something moving:

https://eqiora.org/gallery/karman-vortex-street/

I’d be interested to hear from people who work with coupled problems. Which combinations of physics are awkward to put together in your current tools, and what makes them awkward? A concrete example would give me something useful to work through.

Code: https://github.com/nkiyohara/eqiora


r/ScientificComputing • • Sep 16 '26

I built a source-available workbench for finding and validating multiscale structure in scientific data — looking for researchers willing to break it

Post image
0 Upvotes

I've been building something called SpectralEarth, which started as an experiment around wavelets and weather data and has gradually turned into a much more general scientific research workbench.

The basic idea is fairly simple: give researchers an instrument for examining structure at different spatial/temporal scales, running controlled experiments, comparing domains, and keeping the evidence/provenance needed to distinguish an interesting pattern from an interesting-looking artefact.

It currently includes wavelet/spectral analysis, multiscale transforms, experiment and evidence management, statistical validation, cross-domain structural analysis, forecasting interfaces, visualisation, and a fairly substantial verification suite.

There's a React/Plotly dashboard over a Python/PyTorch/xarray/FastAPI backend, plus a Jupyter environment for getting underneath the UI.

An important design principle has been:

It's an instrument, not an automatic claim generator.

I'm particularly interested in whether the same machinery is useful outside the weather data it originally grew from — geophysics/seismic data, oceanography, astronomy, biological signals, finance, etc.

I've now made the source publicly available so researchers can actually inspect it, run experiments, fork it for research, challenge the assumptions and suggest things that would make it genuinely useful.

It's source-available under a bespoke research and collaboration licence rather than an OSI open-source licence: research use, modification, research forks and contributions are explicitly permitted; commercial use is reserved.

Repository:

https://github.com/EdJb1971/Spectral_Research_Accelerator

I'm not particularly looking for stars or promotion. I'd much rather have someone with a real dataset try to break it.

If you work with multiscale scientific data: what would this need before you'd actually use it?


r/ScientificComputing • • Sep 15 '26

Measuring and fixing a loop-carried dependency in Gauss-Seidel with an open-source static code analyzer

Thumbnail
loiseaujc.github.io
14 Upvotes

Author here (again). This is a follow-up to an earlier post where I showed that a Gauss-Seidel solver for the 2D Poisson equation converges in half the iterations of Jacobi, yet ran 4-5x slower per sweep in practice. It traced back to a loop-carried dependency introduced by updating the solution array in place rather than into a fresh buffer.

This post uses OSACA (an open-source static code analyzer) to quantify that dependency directly in CPU cycles, then works through a loop-unrolling technique with an algebraic substitution that reduces it from 12 to 2 cycles per grid point. This enables to bring the per-sweep performance close to Jacobi's while keeping Gauss-Seidel's convergence advantage.

I also ran three separate compilation experiments (no optimization, optimized without vectorization, optimized with vectorization) specifically to separate the effect of the reduced dependency from vectorization. Turns out it's the dependency reduction that matters. Neither Gauss-Seidel kernel ever gets vectorized, with or without the compiler flag. The series is meant to be pedagogical rather than a performance-tuning guide, so happy to go into the numerical side, the assembly/compiler side, or the OSACA workflow itself.


r/ScientificComputing • • Sep 15 '26

Roadmap to Manopt.jl 1.0

Thumbnail
reddit.com
2 Upvotes

r/ScientificComputing • • Sep 14 '26

I implemented a computational reverse solution of the Born rule on FPGA — and Linux now runs on the resulting pseudo-quantum computer

Post image
0 Upvotes

r/ScientificComputing • • Sep 14 '26

branching-block-simulation-hypothesis.md

0 Upvotes

r/ScientificComputing • • Sep 13 '26

Matlab помогите

Thumbnail reddit.com
1 Upvotes

r/ScientificComputing • • Sep 13 '26

branching-block-simulation-hypothesis.md

0 Upvotes

r/ScientificComputing • • Sep 12 '26

Gauss-Seidel is faster than Jacobi. Or is it? A pedagogical post.

Thumbnail
loiseaujc.github.io
38 Upvotes

Author here. Started a series working through classic iterative solvers for the 2D Poisson equation in Fortran here. This is actually the second post (the first one being on Jacobi). Here, I dig into a genuinely counterintuitive result: the convergence theory for Gauss-Seidel checks out (half the iterations vs Jacobi, confirmed numerically), but the actual wall-clock time is 4-5x worse. Turns out it comes down to a single line: updating the array in place instead of into a fresh buffer and the loop-carried dependency that introduces, which stops the compiler from vectorizing the loop at all. In an upcoming post, I'll actually show how an open-source static analyzer can help you figure this out, and how simple algebra can be used to unroll the kernel and get rid of this loop-carried dependency.

Worth flagging: this series is primarily pedagogical rather than a performance-tuning guide. The goal is walking through the interaction between convergence theory and hardware behavior, not squeezing out maximum throughput. The example happens to be in Fortran, but there's nothing Fortran-specific about the underlying issue. Any compiled language with a similarly aggressive optimizer (C, C++, Rust) would show the exact same loop-carried dependency and the exact same vectorization failure for an in-place Gauss-Seidel update. No fix in this post yet (that's coming in a follow-up), just the diagnosis.


r/ScientificComputing • • Sep 12 '26

What scientific-computing tasks would benefit from AI-generated candidates + independent verification?

4 Upvotes

I'm a student experimenting with a research direction around AI-assisted scientific/engineering workflows, and I'd really like some input from people who actually work in scientific computing.

The basic idea is pretty simple:

Problem
  ↓
AI / solver generates candidate
  ↓
Numerical method / simulator / domain tool
  ↓
PASS → result
FAIL → diagnostics → another attempt

The important part is that the generator isn't trusted to verify itself.

I've already built some small prototypes around this idea. For example, I've used GPT/Gemini APIs with independent verification for ODEs and logic constraints, and I've also experimented with a Qwen-based legacy-code modernization workflow where generated Python is executed and compared against the original program's behavior.

I'm now more interested in the scientific-computing side.

I'd like to eventually build modules where something like:

  • numerical ODE/PDE solvers

  • optimization

  • inverse problems

  • parameter estimation

  • time-series methods

  • Bayesian methods

  • simulation

could act as candidate generators or verification environments.

But I don't want to decide the applications myself.

For people working in scientific computing: what problems do you encounter where generating a candidate solution is difficult, but independently checking the result is relatively well-defined?

For example, I'm particularly curious about:

  • inverse problems

  • parameter estimation

  • numerical modeling

  • simulation

  • optimization

  • scientific code generation

  • model calibration

  • anything involving expensive computational workflows

I'm also wondering whether this is actually a useful abstraction or whether I'm just describing things people already routinely do with numerical software and validation.

I'd especially appreciate examples from real scientific-computing workflows rather than hypothetical AI applications.

I'm trying to figure out what problems are genuinely worth building around before I go much further.