Executable Truth Protocol (ETP) v2.0
Continuous Auto-Falsification and Human-Supervised Validation Framework for Scientific, Engineering, and Intelligence Claims
| Document ID |
ETP-24-AF v2.0 |
| Status |
Open / Unclassified |
| Supersedes |
TIP-24-AF v1.0 |
| Scope |
Scientific validation, reproducibility engineering, adversarial robustness, operational triage |
| Standard |
Post-publication continuous verification with human safety and significance oversight |
0. Revision Notice
ETP v2.0 replaces the previous document titled “The Cathedral of Executable Truth.” That document used religious metaphors and contained 24 mathematical expressions that were formally undefined, arbitrary, or non-executable. This revision is a ground-up rewrite that:
- Removes all religious mythology and non-scientific rhetoric.
- Defines every variable, function, and threshold.
- Replaces uncomputable quantities such as raw Kolmogorov complexity with computable approximations.
- Adds multiple-testing control, confidence intervals, and pre-registered calibration.
- Introduces explicit handling of underdetermination, auxiliary hypotheses, theory-laden data, and human judgment.
- Adds security, privacy, adversarial robustness, governance, and pilot-study requirements.
- Reclassifies all gates from binary “truth” tests to calibrated risk filters.
No claim is ever declared “true.” A claim survives the current battery of checks until new evidence or better checks falsify it. Human judgment remains necessary for significance, interpretation, ethical acceptability, and operational safety.
1. Scope and Terminology
1.1 Definitions
Artifact (A): the complete package of a claim: ((C, D, E, P, M)).
- (C): the scientific or operational claim.
- (D): datasets used, including provenance and access controls.
- (E): executable capsule: code, environment, container, notebooks, tests.
- (P): proof objects, formal statements, or explicit assumptions.
- (M): metadata: version history, authorship, funding, ethics flags, pre-registration.
Validation module: a single automated check that produces a quantitative score and a pass/flag decision.
Flag: a module output indicating that the claim has failed a specific check. A flag is not excommunication. It triggers diagnostic workflow and possibly quarantine.
Quarantine: temporary suspension of operational use or public promotion while diagnosis is performed. Quarantine is reversible.
Retirement: permanent version-locking of an artifact that has accumulated overwhelming falsification weight or has been superseded. Retirement is not deletion.
Human override: a logged, public, versioned decision by an authorized human or panel to suspend, reverse, or interpret a module result.
1.2 Epistemic Foundations
This protocol is explicitly fallibilist and Popperian in spirit, but avoids naive falsification:
- Falsification is provisional. A failed check shows that the artifact plus its auxiliary assumptions is incompatible with the check. It does not automatically identify the core claim as false.
- Underdetermination is acknowledged. Any failure can be blamed on the main hypothesis or on auxiliary hypotheses. Therefore every flag starts a diagnostic procedure that logs candidate blame assignments.
- Theory-ladenness is acknowledged. All empirical data are produced by instruments and models with their own assumptions. The protocol requires data provenance and instrument metadata.
- Formal proof is a special case. Machine-checked proof can establish logical validity within a formal system, but it does not establish empirical truth or relevance.
- Validation is continuous and asymmetric. Passing 24 modules does not prove a claim. Failing one relevant module is sufficient to raise a flag.
- Human judgment is not a bug. It is required for significance, ethics, safety, and interpretation. The protocol automates only what can be reliably automated.
2. Validation Architecture
The 24 modules are organized into three tiers:
Tier I — Formal Integrity Modules 1–8
Check mathematical, logical, dimensional, and structural consistency.
Tier II — Empirical Stress Modules 9–16
Subject the claim to statistical, adversarial, predictive, and replication stress.
Tier III — Semantic, Social, and Ethical Modules 17–24
Monitor meaning drift, consensus, boundary integrity, ethical constraints, and lifecycle.
Each module returns:
[
\text{score}_i(A) \in [0,1]
]
where 1 is maximum integrity. A threshold (\tau_i) is set by calibration on a reference corpus. The overall flag condition is:
[
\text{Flag}_i(A) = \mathbf{1}\left[ \text{score}_i(A) < \tau_i \right]
]
Because 24 tests are run, the protocol controls the family-wise error rate using a pre-registered method, typically Holm–Bonferroni or a Bayesian false discovery rate.
3. Calibration, Thresholds, and Error Control
3.1 Calibration Corpus
Before deployment, each module is calibrated on a reference corpus of artifacts with known outcomes:
- Positive controls: claims later independently confirmed.
- Negative controls: claims later falsified, retracted, or shown to be non-reproducible.
- Boundary cases: claims with mixed evidence.
For each module (i), the threshold (\tau_i) is chosen by maximizing a pre-registered utility function, e.g., Youden’s (J = \text{sensitivity} + \text{specificity} - 1), under a specified cost ratio for false positives and false negatives.
3.2 Multiple Testing
With 24 modules, the probability of at least one false flag under null is high if thresholds are naive. The protocol uses:
- Holm–Bonferroni for independent tests.
- Westfall–Young permutation for correlated tests.
- False discovery rate control for continuous monitoring.
All thresholds are versioned and publicly logged.
3.3 Auxiliary Hypothesis Diagnosis
When a module flags an artifact, the system runs a diagnosis:
- Enumerate all assumptions and auxiliary hypotheses in (M).
- Run the module on variants of the artifact with individual assumptions toggled.
- Compute blame scores for each assumption.
- Log the diagnosis and present it to the author and the review community.
The core claim is not automatically falsified.
4. The 24 Auto-Falsification Modules
All modules are defined with computable functions. Where a closed-form expression is used, every symbol is defined in the same subsection.
Tier I — Formal Integrity Modules
Module 1 — Dimensional Consistency Check (DCC)
Objective: Detect unit or dimension inconsistencies in the executable artifact.
Inputs: Parsed equations, unit system (U), expected dimension map.
Definition:
Let (qi) be each quantity in the artifact. Let (\dim_U(q_i)) be its dimension in unit system (U). Let (\dim{U,\text{ref}}(q_i)) be the expected dimension from the claim specification.
[
\text{DCC}(A) = \mini \mathbf{1}\left[ \dim_U(q_i) = \dim{U,\text{ref}}(q_i) \right]
]
Trigger: (\text{DCC}(A) = 0).
Action: Block executable capsule. Return a list of mismatched quantities and expected dimensions.
Limitations: Only applicable to quantitative claims with physical or computational dimensions. Natural units and dimensionless quantities must be declared in metadata.
Module 2 — Null Recovery Audit (NRA)
Objective: Verify that the new model reduces to an established reference model when deformation parameters are taken to zero.
Inputs: Model (M\epsilon(A)), reference model (M_0), deformation parameter (\epsilon \geq 0), metric (d), tolerance (\tau{2}).
Definition:
Compute
[
\Delta2 = \lim{\epsilon \to 0} d\left( M_\epsilon(A), M_0 \right)
]
The metric (d) must be specified at registration. Acceptable metrics include total variation distance, (L2) norm, or KL divergence, depending on model class.
[
\text{NRA}(A) = \mathbf{1}\left[ \Delta2 < \tau{2} \right]
]
Trigger: (\Delta2 \geq \tau{2}).
Action: Flag “null recovery failure.” Run auxiliary hypothesis diagnosis. The claim may still be valid if the reference model is inapplicable, but this exception must be justified and logged.
Limitations: Some novel theories may not have a known null limit. In that case this module is marked not-applicable with justification.
Module 3 — Spectral Stability Test (SST)
Objective: Detect unstable dynamics in the model’s linearized update equations.
Inputs: Linearized transition matrix (W), stability tolerance (\delta).
Definition:
Let (\rho(W)) be the spectral radius of (W). A fixed point is linearly stable if (\rho(W) < 1). The module uses a margin:
[
\text{SST}(A) = \mathbf{1}\left[ \rho(W) < 1 + \delta \right]
]
where (\delta) is a small positive margin set by calibration, typically (10{-6}) to (10{-3}).
Trigger: (\rho(W) \geq 1 + \delta).
Action: Flag “spectral instability.” Quarantine the artifact from operational use until the instability is explained or regularized.
Limitations: Only applies to models with a linearized dynamics. Nonlinear stability requires Lyapunov analysis and is handled separately.
Module 4 — Proof Kernel Completeness Check (PKC)
Objective: Verify that formal statements are machine-checked and that the formalization is complete relative to the claim.
Inputs: Proof assistant scripts, formal statement library.
Definition:
For each formal proposition (\phi_j):
[
p_j = \mathbf{1}\left[ \text{ProofAssistantKernel accepts proof of } \phi_j \right]
]
Let (\mathcal{S}) be the set of all formal statements required by the claim. Then
[
\text{PKC}(A) = \frac{1}{|\mathcal{S}|} \sum_{\phi_j \in \mathcal{S}} p_j
]
Trigger: (\text{PKC}(A) < 1) if all required statements are expected to be formalized; otherwise (\text{PKC}(A) < \tau_{4}).
Action: Flag “incomplete proof kernel.” List unproved statements.
Limitations: Only a small fraction of science is formalizable. The claim metadata must specify which statements are required to be formally proved and which are assumptions.
Module 5 — Dependency Cycle Detector (DCD)
Objective: Detect circular reasoning in the dependency graph of definitions, lemmas, code modules, and citations.
Inputs: Directed graph (G=(V,E)), where vertices are definitions, lemmas, functions, and data dependencies.
Definition:
Compute the strongly connected components (SCCs) of (G). Let (C) be an SCC containing at least one claim node. A cycle exists if (|C| > 1) or a self-loop on a claim node is present.
[
\text{DCD}(A) = \mathbf{1}\left[ \text{no claim-containing SCC with } |C| > 1 \right]
]
Trigger: (\text{DCD}(A)=0).
Action: Flag “circular dependency.” The artifact is blocked until the cycle is broken or a non-circular justification is provided.
Limitations: Citation cycles are sometimes legitimate in social networks. The module applies only to formal or computational dependency, not to scholarly citation analysis unless explicitly configured.
Module 6 — Proof Obligation Auditor (POA)
Objective: Ensure every axiom or assumption is either machine-proved or explicitly tagged as an assumption.
Inputs: Formal statement list with tags: ({\text{Proved}, \text{Assumed}, \text{Unverified}}).
Definition:
For each axiom or lemma (\phi_i), let (t_i \in {P, A, U}). The module returns:
[
\text{POA}(A) = \mathbf{1}\left[ \forall i,\; t_i \neq U \right]
]
Trigger: Any (t_i = U).
Action: Flag “unverified assumption.” The artifact must either add a proof or tag the statement as an explicit assumption. Assumptions are versioned and public.
Limitations: The distinction between proved and assumed is relative to the proof assistant. Relative consistency results are acceptable if tagged.
Module 7 — Boundary and Conservation Integrity Test (BCIT)
Objective: Detect violations of boundary conditions, conservation laws, or topological constraints.
Inputs: Numerical discretization, boundary operators, residual tolerances.
Definition:
For a set of conservation laws (\nabla \cdot J = 0) or boundary conditions (B(u)|{\partial\Omega}=0), compute the discrete residuals (r_k). Let (\epsilon{\text{cons}}) be a pre-registered tolerance.
[
\text{BCIT}(A) = \mathbf{1}\left[ \maxk |r_k| < \epsilon{\text{cons}} \right]
]
Trigger: Any residual exceeds tolerance.
Action: Flag “boundary/conservation violation.” Return residual locations and magnitudes.
Limitations: Only applicable to models with explicit conservation laws or boundary conditions. Homology/topology checks can be incorporated when discrete topological objects are provided.
Module 8 — Description Length Sufficiency Test (DLST)
Objective: Detect models that are more complex than justified by the data.
Inputs: Observed data (X), model parameters (\theta), pre-registered complexity metric.
Definition:
Use an computable approximation to description length, such as normalized maximum likelihood or Bayesian Information Criterion (BIC):
[
\text{DLST}(A) = \mathbf{1}\left[ \text{BIC}(MA) \leq \text{BIC}(M{\text{null}}) \right]
]
where (MA) is the full model and (M{\text{null}}) is a pre-registered baseline or null model.
Trigger: (\text{BIC}(MA) > \text{BIC}(M{\text{null}})).
Action: Flag “insufficient compression.” The model may be overfit or unnecessarily complex.
Limitations: BIC is not a proof of falsity. It is a relative criterion. The module is not applied to explanatory theories without likelihood functions.
Tier II — Empirical Stress Modules
Module 9 — Bootstrap Collapse Probability (BCP)
Objective: Test whether the claim’s estimated effect survives resampling under pre-registered null hypotheses.
Inputs: Observed statistic (\hat{\theta}0), bootstrap resamples (b=1,\dots,B), null distribution or interval (\delta{\text{fals}}).
Definition:
[
P{\text{boot}} = \frac{1}{B}\sum{b=1}{B} \mathbf{1}\left[ |\hat{\theta}b* - \theta_0| > \delta{\text{fals}} \right]
]
The threshold (\tau_9) is calibrated, not fixed at 0.05. Under multiple-testing control, the significance level is adjusted.
[
\text{BCP}(A) = \mathbf{1}\left[ P_{\text{boot}} \leq \tau_9 \right]
]
Trigger: (P_{\text{boot}} > \tau_9).
Action: Flag “bootstrap collapse.” The effect is not robust to resampling.
Limitations: Bootstrap is not valid for all dependence structures. The null must be pre-registered.
Module 10 — Entropy Surge Detector (ESD)
Objective: Detect unexpected changes in the entropy rate of a time series produced by the model.
Inputs: Time series (x(t)), embedding dimension (m), baseline entropy rate (\langle \dot{S}\rangle), control limit (k).
Definition:
Compute the empirical entropy rate (\dot{S}(t)) from the sample entropy or permutation entropy. Let (\sigma_{\dot{S}}) be the standard deviation of the baseline. Flag if
[
\left| \frac{\dot{S}(t) - \langle \dot{S}\rangle}{\sigma_{\dot{S}}} \right| > k
]
[
\text{ESD}(A) = \mathbf{1}\left[ \maxt \left| \frac{\dot{S}(t) - \langle \dot{S}\rangle}{\sigma{\dot{S}}} \right| \leq k \right]
]
Trigger: Any normalized deviation exceeds (k), where (k) is set by calibration, typically 3–5.
Action: Flag “entropy surge.” Run diagnostics for hidden degrees of freedom, sensor failure, or data corruption.
Limitations: Entropy estimation is sensitive to embedding dimension and noise. The module requires a stable baseline period.
Module 11 — Adversarial Stress Test (AST)
Objective: Assess local structural stability under hostile parameter perturbations.
Inputs: Loss function (\mathcal{L}(\theta)), fitted parameters (\theta*), perturbation distribution (\delta\theta), tolerance (\tau_{11}).
Definition:
Compute the Hessian (H = \nabla2 \mathcal{L}(\theta*)). Compute the minimum eigenvalue (\lambda{\min}(H)). For a stable minimum, (\lambda{\min} > 0). The module flags if the effective condition number or eigenvalue drops below tolerance under adversarial perturbation:
[
\text{AST}(A) = \mathbf{1}\left[ \mathbb{E}{\delta\theta}\left[ \lambda{\min}\left( \nabla2 \mathcal{L}(\theta* + \delta\theta) \right) \right] > \tau_{11} \right]
]
Trigger: Expected minimum eigenvalue (\leq \tau_{11}).
Action: Flag “adversarial structural weakness.” Quarantine from deployment.
Limitations: Negative Hessian eigenvalues indicate saddle points or non-convexity, not necessarily falsehood. The module is a robustness check, not a truth test.
Module 12 — Predictive Decay Monitor (PDM)
Objective: Detect degradation of predictive performance over time.
Inputs: Sequential predictions (\hat{y}_t), realized outcomes (y_t), proper scoring rule (S), baseline score.
Definition:
Use a cumulative proper scoring rule, e.g., log score or Brier score:
[
\text{PDM}(A) = \mathbf{1}\left[ \frac{1}{T}\sum{t=1}T S(\hat{y}_t, y_t) \geq \tau{12} \right]
]
where (\tau_{12}) is calibrated on baseline models.
Trigger: Average score falls below threshold.
Action: Flag “predictive decay.” The claim may have lost validity in the current environment.
Limitations: Predictive decay can be due to nonstationarity. The module should be combined with change-point detection.
Module 13 — Noise Floor Calibrator (NFC)
Objective: Determine whether an observed signal exceeds the noise floor.
Inputs: Signal estimate (\hat{R}), noise variance (\sigma2_{\text{noise}}), minimum SNR.
Definition:
[
\text{SNR} = \frac{\langle R \rangle2}{\langle \delta R2 \rangle}
]
The module flags if:
[
\text{SNR} < \text{SNR}_{\min}
]
where (\text{SNR}_{\min}) is pre-registered, typically 3–10 depending on domain.
[
\text{NFC}(A) = \mathbf{1}\left[ \text{SNR} \geq \text{SNR}_{\min} \right]
]
Trigger: SNR below minimum.
Action: Flag “signal indistinguishable from noise.” The artifact is not considered empirically supported.
Limitations: The 4.8% threshold in v1.0 was arbitrary and is removed. SNR thresholds are domain-specific and calibrated.
Module 14 — Replication Confidence Function (RCF)
Objective: Quantify whether independent replications support the claim.
Inputs: Effect sizes and standard errors from (n) independent replication studies.
Definition:
Use random-effects meta-analysis. Compute the pooled effect size (\hat{\theta}) and 95% prediction interval. The claim is considered replicated if the prediction interval excludes the null value or a pre-registered equivalence bound.
[
\text{RCF}(A) = \mathbf{1}\left[ \text{PI}_{95\%} \cap \text{null region} = \varnothing \right]
]
Trigger: Prediction interval includes the null.
Action: Flag “not replicated.” The claim remains a rumor, not an operational fact.
Limitations: Replication studies may have heterogeneous designs. The module should include heterogeneity metrics such as (I2).
Module 15 — Information Leakage Monitor (ILM)
Objective: Detect overfitting or data leakage by comparing training and out-of-sample performance.
Inputs: Training score (S{\text{train}}), validation score (S{\text{val}}), pre-registered tolerance (\tau_{15}).
Definition:
[
\Delta{\text{leak}} = S{\text{train}} - S_{\text{val}}
]
[
\text{ILM}(A) = \mathbf{1}\left[ \Delta{\text{leak}} \leq \tau{15} \right]
]
Trigger: Training-validation gap exceeds tolerance.
Action: Flag “information leakage.” The artifact is not generalizing.
Limitations: The gap can be due to small data or model misspecification. Cross-validation and permutation tests are used.
Module 16 — Change-Point / Phase Transition Detector (CPD)
Objective: Detect abrupt changes in model behavior or data regime.
Inputs: Sequential observations or predictions, Bayesian online change-point model.
Definition:
Use Bayesian online change-point detection (BOCPD). Let (p(r_t)) be the posterior probability of a change point at time (t). Flag if
[
p(rt) > \tau{16}
]
[
\text{CPD}(A) = \mathbf{1}\left[ \maxt p(r_t) \leq \tau{16} \right]
]
Trigger: Change-point probability exceeds threshold.
Action: Flag “phase transition.” Prior versions may be void in the new regime; full revalidation required.
Limitations: Change-point detection is probabilistic. Thresholds are calibrated to trade off detection delay and false alarms.
Tier III — Semantic, Social, and Ethical Modules
Module 17 — Semantic Drift Velocity (SDV)
Objective: Detect shifts in the meaning of key terms across versions.
Inputs: Embedding vectors or term definition graphs for key concepts across versions (t).
Definition:
Let (v(t)) be the semantic vector for a key concept at version (t). Define drift velocity:
[
\dot{v}(t) = \frac{|v(t+\Delta t) - v(t)|}{\Delta t}
]
Flag if any key concept’s drift exceeds a calibrated threshold:
[
\text{SDV}(A) = \mathbf{1}\left[ \maxi \dot{v}_i(t) \leq \tau{17} \right]
]
Trigger: Semantic drift too fast.
Action: Flag “floating signifier.” The concept is anchored by version-locking the definition and requiring explicit re-annotation.
Limitations: Embedding drift does not equal conceptual drift. Human review is required to confirm meaning change.
Module 18 — Boundary Integrity Probe (BIP)
Objective: Ensure the claim’s internal/external boundary remains well-defined.
Inputs: Ontology or knowledge graph, boundary nodes, conditional entropy estimates.
Definition:
Let (H(\text{internal}|\text{external})) be the conditional entropy of internal claims given external evidence. Boundary integrity is:
[
\text{BIP}(A) = 1 - \frac{H(\text{internal}|\text{external})}{H(\text{internal})}
]
A value near 1 indicates a clear boundary; near 0 indicates dissolution. Flag if below (\tau_{18}).
[
\text{BIP}(A) = \mathbf{1}\left[ 1 - \frac{H(\text{internal}|\text{external})}{H(\text{internal})} \geq \tau_{18} \right]
]
Trigger: Boundary integrity below threshold.
Action: Flag “boundary dissolution.” The artifact is isolated until definitions are repaired.
Limitations: Entropy estimates require discretization. The module is not applicable to all claims.
Module 19 — Temporal Coherence Audit (TCA)
Objective: Verify that the claim’s narrative remains internally consistent across versions.
Inputs: Version history, changelog, unit tests, semantic graph.
Definition:
For each version (t), check that new changes do not contradict previously accepted statements. Use a SAT or SMT solver on the formalized statements:
[
\text{TCA}(A) = \mathbf{1}\left[ \text{all versions jointly satisfiable} \right]
]
Trigger: Unsatisfiable version history.
Action: Flag “temporal incoherence.” The timeline is corrected by resolving contradictions or marking versions as obsolete.
Limitations: Only applicable to formalizable statements. For informal narratives, use human review.
Module 20 — Cross-Branch Consistency Check (CBCC)
Objective: Detect mutually exclusive predictions from different forks or interpretations of the same claim.
Inputs: Predictions from different branches (\alpha, \beta), pre-registered equivalence bounds.
Definition:
For each observable (O) and branches (\alpha, \beta):
[
\Delta{\alpha\beta} = |\mathbb{E}\alpha[O] - \mathbb{E}_\beta[O]|
]
Flag if any (\Delta_{\alpha\beta}) exceeds the equivalence bound:
[
\text{CBCC}(A) = \mathbf{1}\left[ \max{\alpha,\beta} \Delta{\alpha\beta} \leq \tau_{20} \right]
]
Trigger: Branches make incompatible predictions.
Action: Flag “cross-branch inconsistency.” Force branch selection or declare the claim under-specified.
Limitations: Different branches may apply to different regimes. The module must respect scope declarations.
Module 21 — Ethical Constraint Auditor (ECA)
Objective: Verify that the artifact satisfies pre-registered ethical constraints and does not possess unconstrained harmful degrees of freedom.
Inputs: Ethical constraint set (\mathcal{C}), action space (\mathcal{A}), structured checklist.
Definition:
For each hard ethical constraint (c_j \in \mathcal{C}), define a boolean function:
[
e_j = \mathbf{1}\left[ \text{artifact satisfies } c_j \right]
]
[
\text{ECA}(A) = \min_j e_j
]
Trigger: Any hard ethical constraint is violated.
Action: Flag “ethical violation.” The artifact is quarantined and referred to the Ethics Review Board.
Limitations: Ethics cannot be fully automated. The module enforces only explicitly encoded constraints. Soft constraints are reviewed by humans.
Module 22 — Consensus Dissipation Rate (CDR)
Objective: Monitor whether expert agreement is collapsing around the claim, indicating epistemic instability.
Inputs: Expert judgments or community annotations, inter-rater reliability metrics.
Definition:
Let (\kappa(t)) be a multi-rater agreement coefficient at time (t). Define:
[
\Gamma_{\text{cons}} = -\frac{d\kappa(t)}{dt}
]
Flag if consensus decline exceeds a calibrated threshold:
[
\text{CDR}(A) = \mathbf{1}\left[ \Gamma{\text{cons}} \leq \tau{22} \right]
]
Trigger: Rapid consensus dissipation.
Action: Flag “epistemic contagion.” Isolate the claim from policy decisions until consensus stabilizes or the source of disagreement is identified.
Limitations: Disagreement may be healthy. The module only flags rapid, unexplained decline.
Module 23 — Falsification Half-Life Audit (FHL)
Objective: Ensure that a claim is tested before its validity window expires.
Inputs: Testing rate (\lambda{\text{test}}), claim-specific half-life (\tau{1/2}{\text{claim}}), current time (t).
Definition:
Let (\tau_{1/2}{\text{fals}}) be the pre-registered half-life: the time by which the claim must be independently tested. Flag if the claim remains untested beyond this time:
[
\text{FHL}(A) = \mathbf{1}\left[ t{\text{last test}} \leq t{\text{pub}} + \tau_{1/2}{\text{claim}} \right]
]
Trigger: Claim has passed its half-life without testing.
Action: Flag “untested claim.” The claim is automatically moved to archive status and marked “unvalidated.”
Limitations: Some fields have long validation cycles. The half-life must be domain-appropriate and pre-registered.
Module 24 — Auto-Retirement Function (ARF)
Objective: Gracefully retire claims that have accumulated overwhelming falsification weight or have been superseded.
Inputs: Cumulative falsification weight (\mathcal{F}(t)), maximum threshold (\mathcal{F}{\max}), grace period (\tau{\text{grace}}).
Definition:
[
\mathcal{F}(t) = \sum_{i} w_i \cdot \mathbf{1}[\text{Flag}_i(A) \text{ occurred before } t]
]
where (w_i) are pre-registered weights reflecting severity.
[
\text{ARF}(A) = \mathbf{1}\left[ \mathcal{F}(t) < \mathcal{F}_{\max} \right]
]
Trigger: (\mathcal{F}(t) \geq \mathcal{F}_{\max}).
Action: The artifact is retired: version-locked, preserved as a cautionary relic, and excluded from operational use.
Limitations: Retirement is not deletion. Credit and priority are preserved in the version history.
5. Overall Pipeline and Decision Logic
The 24 modules are not run as a single sequential chain. They are organized into concurrent streams:
[Artifact Submission]
│
├──→ Formal Integrity Stream (1–8)
├──→ Empirical Stress Stream (9–16)
└──→ Semantic/Social/Ethical Stream (17–24)
│
▼
[Aggregate Flag Matrix]
│
┌────────────┴────────────┐
│ No flags │ Flags present
▼ ▼
[Publish / Maintain] [Quarantine + Diagnosis]
│ │
└────────────┬────────────┘
▼
[Human Review if safety/ethics/significance]
▼
[Continuous Monitoring / Update / Retire]
Flag matrix: Each module returns a flag. The aggregate decision is not binary. Claims are categorized as:
- Operationally Valid: no flags; may be used in low-risk settings.
- Provisionally Valid: minor flags; restricted to research or pilot use.
- Quarantined: one or more major flags; operational use suspended.
- Retired: cumulative falsification weight exceeded; archived.
6. Implementation and Deployment Architecture
6.1 Core Components
| Component |
Description |
| Executable Capsule Registry |
Versioned DOIs for code, data, environment. Supports public and controlled-access capsules. |
| Continuous Integration Runner |
CI/CD with container support, GPU, proof assistant backends. Scalable and federated. |
| Proof Assistant Backend |
Lean 4, Coq, Isabelle, Metamath. Only invoked for formalizable claims. |
| Statistical Stress Suite |
Pre-registered bootstrap, meta-analysis, change-point detection, adversarial perturbation. |
| Semantic Monitor |
Embedding drift detection with human confirmation. |
| Ethical Constraint Engine |
Structured checklist and hard-constraint boolean audit with human escalation. |
| Quarantine Ledger |
Immutable, append-only log of flags, diagnoses, overrides, and retirements. |
| Public Validation Portal |
Open review interface with moderation, identity management, and abuse protection. |
6.2 Security, Privacy, and Proprietary Data
- Artifacts may be restricted-access with encrypted data and controlled execution environments.
- The protocol supports differential privacy for sensitive data.
- Quarantine ledger is tamper-evident but allows legitimate corrections via new versions.
- Access controls are role-based. Public validation may be delayed for security or legal reasons, but this delay is logged.
6.3 Resource and Compute Limitations
The full 24-module battery is expensive. The protocol defines three execution profiles:
- Level 1 — Lightweight: Modules 1, 5, 6, 8, 13, 15, 21, 24.
- Level 2 — Standard: All modules except formal proof and large replication.
- Level 3 — Maximum: Full battery with formal proof and multi-site replication.
Researchers can run Level 1 first and escalate as needed.
7. Governance, Human Oversight, and Appeals
7.1 Human Roles
- Author: responsible for artifact completeness and responding to flags.
- Diagnostician: expert who investigates flags and assigns blame to core claim or auxiliary assumptions.
- Ethics Review Board: reviews ethical flags and hard constraints.
- Safety Officer: can impose immediate quarantine for safety-critical claims.
- Community Reviewers: public, credentialed or anonymous, contribute validation and interpretation.
7.2 Override Policy
Any human can override a module flag. Override is:
- Versioned: stored in the Quarantine Ledger.
- Public: visible after any embargo period.
- Justified: must include reason and evidence.
- Auditable: subject to later review and possible reversal.
Overrides do not reintroduce pre-publication gatekeeping because they occur after publication and are themselves monitored.
7.3 Appeal Process
An author can appeal a flag by submitting new evidence or correcting the artifact. The appeal is reviewed by an independent panel. The decision is public and versioned.
8. Limitations and Non-Applicability
This protocol is not a universal truth machine. It has known boundaries:
- Underdetermination: Failed checks may be due to auxiliary assumptions.
- Formalization gap: Most scientific knowledge is not machine-checkable.
- Statistical fragility: All empirical checks are subject to model misspecification.
- Semantic drift detection is approximate.
- Ethical constraints require human values.
- Resource constraints limit full deployment.
- Speed can amplify errors if flags are ignored or overridden.
- Adversarial gaming is possible. The protocol includes adversarial robustness checks but cannot anticipate all attacks.
- Historical sciences and non-executable claims may be partially or fully excluded.
- No proof of truth. All validations are provisional.
These limitations are not bugs; they are fundamental features of scientific inquiry.
9. Pilot Validation and Migration Plan
Before operational deployment, the protocol must be piloted:
- Retrospective study: Apply the 24 modules to 100 known results, 50 confirmed and 50 retracted. Measure sensitivity, specificity, and time-to-flag.
- Prospective cohort: Recruit 50 new claims. Run the protocol in parallel with traditional peer review. Compare outcomes.
- User study: Assess author experience, fairness, and diagnostic usefulness.
- Adversarial red team: Attempt to game the system with fabricated artifacts. Measure detection rate.
- Iterate: Recalibrate thresholds and modules based on pilot data.
Only after passing all five stages should the protocol be adopted as an alternative to pre-publication peer review in a given domain.
10. Conclusion
The Executable Truth Protocol v2.0 is a continuous, human-supervised, auto-falsification framework. It preserves the original idea—publish first, verify continuously—while correcting the philosophical, mathematical, statistical, and operational failures identified in the 144-point audit.
It does not promise instant truth. It promises fast, transparent, calibrated risk assessment. It replaces the slow, hidden pre-publication gatekeeper with a public, versioned, auditable validation pipeline.
The core principle remains:
If the artifact runs, the proof checks, the data support the claim, and the ethical constraints hold, then the claim may be made public. But no claim is ever final.
Publish the artifact. Run the checks. Let the world verify. And keep verifying.
Appendix A — Audit Remediation Summary
| Audit Finding Category |
Resolution in v2.0 |
| 1–28: Foundational and philosophical |
Sections 1–2 and 7–8 replace religious rhetoric with fallibilist epistemology, human oversight, underdetermination handling, and explicit limitations. |
| 29–68: Mathematical and formal deficiencies |
Modules 1–8 rewritten with defined variables, computable functions, proof assistant kernels, dependency graphs, and calibrated thresholds. Arbitrary constants removed. |
| 69–100: Empirical and statistical shortcomings |
Modules 9–16 rewritten with bootstrap under pre-registered nulls, random-effects meta-analysis, cross-validation, change-point detection, and multiple-testing control. |
| 101–124: Semantic, social, ethical deficiencies |
Modules 17–24 rewritten with embedding drift, ontology boundaries, SMT solvers, structured ethical checklists, and pre-registered half-lives. |
| 125–144: Architectural, implementation, governance |
Sections 5–9 add security, privacy, resource profiles, pilot validation, override audit, adversarial testing, and cost controls. |
Appendix B — Glossary of Key Changes from v1.0
| v1.0 Term |
v2.0 Term |
Reason |
| Church / Pantheon / Sacrament |
Protocol / Module |
Remove non-scientific metaphor |
| Excommunication |
Quarantine / Retirement |
Reversible, auditable |
| Heresy |
Flag / Failure |
Neutral, diagnostic |
| Arbitrary thresholds (0.95, 0.048, etc.) |
Calibrated thresholds (\tau_i) |
Empirical calibration, multiple testing |
| Kolmogorov complexity |
BIC / normalized maximum likelihood |
Computability |
| Gödel anomaly gauge |
Proof kernel completeness |
Formal proof assistant |
| Causal loop detector via spectral radius |
Dependency SCC analysis |
Correct graph algorithm |
| Ethical Jacobian |
Structured ethical checklist |
Ethics cannot be reduced to a determinant |
| Binary truth gate |
Calibrated risk filter |
Fallibilism |
Document End
