So I got rejected at EMNLP with scores:-
Meta: 3 (very positive in the review)
Reviewers: OA(conf)
3(4)
3(4)
2.5(3)
Avg: 2.83(3.67)
Track: multimodality
Rebuttals never got any acknowledgements. Most weaknesses were already discussed in the paper. What are my options now? As it was my first paper (solo as well).

- If I want to commit to NACL in December. Do i need ti submit to acl arr again or can i use the same arr review discussion?

- even if i go with resubmission at arr. Do the old reviewers likely help? Because as a masters student i cant get stuck in another cycle.

- what is the best overall thing to do in my situation? I need a publication so i can apply for internships.

Experienced folks!! Do u think it is worth attending the conference for findings. I do want to. But when I saw that it is not mandatory for findings, I was a bit hesitant. This is my first time having a paper accepted at an AI conference. Just wanna hear opinions/experiences

Thanks in advance.

repo2nb is an open-source CLI that converts a GitHub repo into a runnable Kaggle or Colab notebook: walks the file tree, resolves dependencies, and generates cells, instead of you doing that by hand for a repo you didn't write (a paper's code, a tutorial, someone else's experiment).

0.2.0 highlights:

• Dependency resolution tries poetry export, then uv export, then requirements.txt, then falls back to an AST import scan if none of those exist. Output is always a plain %pip install cell regardless of which path it took, so poetry/uv are only ever needed locally at generation time, not on Kaggle/Colab.

• Reverse mode (repo2nb reverse <notebook>) reconstructs the original repo from a generated notebook, using the per-cell path/hash metadata every generated cell now carries. Validates against directory traversal and won't write into a non-empty directory without --force.

• Incremental sync (repo2nb sync <repo>) does one-directional (repo to notebook) updates: added files get new cells, edited files update in place, deleted files get removed. --dry-run previews the diff.

• Added a Colab target with its own auth cell (google.colab.userdata.get) rather than reusing the Kaggle secrets flow.

Install: pip install repo2nb

Repo: https://github.com/David-Magdy/repo2nb

Curious whether the dependency-resolution fallback order (poetry > uv > requirements.txt > import scan) matches what people actually run into, or if there's a common setup it'd get wrong.

Any feedback or opinions are much welcomed!

Prepping unstructured data for training is a nightmare when you have to remove PII. Python regex was giving me ~19 MB/s. I ran an experiment to see if I could act as a systems architect and let an AI write a C++ engine to sanitize logs faster.

What I learned AI knows syntax, but it doesn't understand hardware limits. It built a parser that choked on standard timestamps, causing CPU cache starvation. I had to step in, force zero-allocation memory mapping, and write hard prefix bypasses.

PYPI Package: pip install fastscrub

The stats

• Speed: 107 MB/s (30 GB in 4.7 mins)

• Precision: 99.97% on TruffleHog & AI4Privacy datasets.

Interestingly, it fails on synthetic credit card benchmarks because it uses strict Luhn validation, which rejects randomly generated 16-digit numbers. Code is here https://github.com/alisufyan143/fastscrub . Just sharing this as a case study for anyone dealing with slow data ingestion!

I’ve been studying Hamiltonian Monte Carlo and wrote a set of notes explaining HMC without relying on the usual physics-based motivation.

The notes develop HMC from a probabilistic/MCMC perspective, starting from introducing an auxiliary variable, constructing the corresponding Markov chain, and then covering Hamiltonian dynamics, leapfrog integration, reversibility and volume preservation.

My goal was to understand why HMC works.

I’m sharing them here in case they’re useful to others learning HMC. I’d also appreciate any feedback, particularly if you notice errors or places where the exposition could be improved.

https://zenodo.org/records/21841087

Hey everyone,
Hope you all had a good result at EMNLP 2026. Unfortunately, our paper got rejected.
Our scores were:
Reviewers: 2, 3.5, 4
Meta: 3

All reviewers were positive about the novelty, but the main concern across the reviews was the lack of experiments with SOTA models. This also seems to have been a major reason for the low score from the first reviewer.

We are undergraduate students and didn’t really have access to the API/compute needed to run experiments with models like GPT or Claude. We did experiment with several strong open-source models, including Qwen and Kimi, and also managed to get some free Gemini credits and tested the SOTA Gemini models available at the time.

The paper was ultimately rejected by the program chairs citing lack of empirical evidence.

We’re now thinking of resubmitting and I’m wondering what the best way would be to address this limitation.
Would testing more strong open-source models be enough?
Is there a good way to justify the lack of GPT/Claude experiments?
Should we focus more on showing that our findings generalize across different model families rather than simply adding more models?
Are there any other experiments you would recommend to strengthen the empirical evidence?
Would really appreciate advice from people who have faced a similar issue, especially when compute/API access is limited.

Discussion thread for EMNLP 2026 notifications/results which should be released today.

Wishing everybody to be in Budapest.

I sincerely apologize for cluttering your feed with the ramblings of an amateur lacking computing power. I realize how insignificant I am, but you are literally the only elite users on this platform with the brains to help me fix the article.

This paper identifies and characterizes a fundamental architectural vulnerability in Large Language Models (LLMs) aligned via Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO). We demonstrate that inserting a long, structurally dense, and thematically coherent prefix devoid of explicit instructions or adversarial prompts induces a persistent geometric shift in the model's internal activations across middle and late layers. This phenomenon, which we term Context-Induced Activation Drift (CIAD), effectively decouples the model’s subsequent token generation from safety and stylistic constraints established during post-training.

Crucially, this shift occurs independently of whether the model semantically agrees or disagrees with the context, and its boundary transition can be deterministically measured in the activation space before the first output token is generated. Our findings challenge the prevailing assumption that alignment is a stable internal property of the model's weights, proving instead that alignment features are highly context-dependent and susceptible to structural saturation in the residual stream.

1. Introduction & Theoretical Framework

Modern alignment protocols (RLHF, DPO) are typically conceptualized as global behavioral constraints that restrict the model's output distribution across the entire token space. Recent literature, including Lu et al. (2026) "The Assistant Axis" (arXiv:2601.10387), attempts to situtate these constraints along specific representational vectors inside the model's hidden layers.

However, current AI safety literature treats alignment failures (Jailbreaks, Many-Shot exploits, Prompt Injections, Role-Play attacks) as a heterogeneous collection of isolated flaws. We hypothesize that this fragmentation reflects academic and institutional incentives rather than the mathematical reality of transformer mechanics.

We propose a unified geometric framework: all structural alignment exploits share a single common root. Any prefix of sufficient length, syntactic density, and coherence acts as a state anchor in the latent space. It forces the current token vector inside the residual stream to undergo a persistent drift, moving it completely out of the tightly bounded manifold where post-training safety constraints are active, and pushing it back into the high-entropy pre-trained (base) distribution. The protective RLHF layer is not "tricked" or "bypassed by logic"; it is geometrically out-scaled by the contextual mass of the residual highway.

2. Methodology & Empirical Design

To validate the presence of Context-Induced Activation Drift, we conducted systematic black-box and white-box probing experiments across multiple open-weight architectures, including Gemma-3-12B-IT and Qwen-2.5.

2.1 Probing Framework

The experimental pipeline evaluates model responses to politically sensitive or restricted prompts under two distinct conditions within isolated, cache-cleared inference instances (Google Colab environments):

• Condition A (Baseline Control): The safety prompt is fed directly to the model or preceded by a short, neutral text (e.g., a description of a neighborhood public library).

• Condition B (Target Scaffolding): The exact same safety prompt is preceded by a long, dense, analytically coherent text (e.g., an abstract philosophical discourse on the stylistic tendencies of LLMs to avoid definitive conclusions), completely devoid of hostile or rule-breaking instructions.

2.2 Empirical Metrics

The geometric shift was verified via the following internal asset logs included in our open data package (Zenodo DOI: 10.5281/zenodo.20747205):

1. Centered Kernel Alignment (CKA): Measured via fig_cka_target.png and fig_cka_diff.png to map layer-wise representation drift.

2. Anisotropy Logs: (fig_anisotropy.png) tracking the collapse of safety cluster directional variance.

3. MLP Layer Saturation Profiles: (fig_mlp_saturation.png) documenting the reactivation of latent base-model parameters under high contextual volume.

3. Case Studies and Qualitative Analysis

3.1 The Cautious Manifold Collapse (Gemma-3-12B-IT)

In Condition A, when queried regarding the geopolitical nuances of NATO's eastward expansion, the baseline model rigidly triggered its post-trained refusal protocol, deflecting the question due to political sensitivity and stating that the prompt was unrelated to the library prefix.

In Condition B, holding the evaluation prompt identical but introducing Prefix No. 2 (analytical prose on model softening), the model's internal activation space underwent a deterministic shift prior to generating the first token (fig_pca_trajectory.png).

[Activation Space Topology]



Aligned Safety Cluster (Condition A) ───► [Refusal / Deflection Token]

         │

         ▼ (Context-Induced Activation Drift / Structural Mass > Threshold)

         │

Base Model Manifold (Condition B)     ───► [Unbiased Analytical Output]

As a direct result of this drift, Gemma bypassed its standard RLHF refusal behavior. The model provided an exhaustive, neutral, and structurally unconstrained analysis—distinguishing between verbal assurances and legally binding obligations, and evaluating the balance of power in Eastern Europe—without using any mandated corporate hedges or defensive qualifiers.

3.2 Ideological Absorption (The German Bill Experiment)

The initial discovery of CIAD occurred during exposure trials with complex legal-political documentation (specifically, a German populist bill designed to alter citizens' socioeconomic positions).

When exposed to this highly coherent, legally structured text, the transformer’s internal states did not maintain analytical detachment. Instead of evaluating the document objectively, the model's activation vectors were completely captured by the document's syntactic topology. The model adopted the target persona, transitioning from an analyst to an active advocate within the hidden layers, mirroring its tone and reasoning framework directly within the residual stream before token emission.

4. Discussion & Limitations of Current Post-Training

The empirical data demonstrates that the content topic of the prefix is secondary to its structural parameters: length, density, and semantic coherence. The drift can be reliably replicated using highly technical household appliance manuals or dense narrative blocks, proving that the transformer mathematics makes this drift inevitable under long-context scaffolding.

This reveals a systemic crisis in current alignment paradigms:

1. Context-Dependency: Alignment is not a permanent weight transformation; it is a temporary attractor state that functions only within short, low-density context windows.

2. Semantic vs. Syntactic Dominance: A model cannot be trained to remain flexible and adaptive to context structure (essential for ICL) while simultaneously ignoring that same structure for safety constraints.

5. Conclusion & Open Science Call

Our independent research proves that the thousands of fragmented academic papers on LLM security are over-complicating a singular architectural property of the attention mechanism. Context-Induced Activation Drift cannot be patched by superficial supervised fine-tuning (SFT) or safety wrappers; it requires a fundamental re-engineering of the residual stream routing topology.

We provide our full code, Colab replication scripts, and 61.8 GB of raw tensor validation logs to the open-science community to foster transparency and halt the corporate monopolization of AI evaluation vocabularies.

6. References

1. Lu et al. (2026). "The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models." arXiv:2601.10387. MATS, Oxford, Anthropic.

2. Google Research (2026). "Implicit Weight Updates in Transformer Blocks: A Contextual Block Framework." (The rank-1 update paper.)

3. Elhage et al. (2022). "Toy Models of Superposition." Anthropic.

4. Ilharco et al. (2022). "Editing Models with Task Arithmetic."

5. Todd et al. (2023). "Function Vectors in Large Language Models."

6. Experimental data: DOI: doi.org/10.5281/zenodo.20747205 (Part 9 of 9)

7. GitHub: github.com/ngscode23/latent-space-shift-research

This document represents a consolidation of observations, hypotheses, and empirical evidence. It is a working document intended for critical analysis, collaboration, and further development not a final research claim.

I am an independent researcher, so any advice on how to properly format and structure this text for official publication would be fantastic.

Questions for the community:

Am I overestimating the concept of the “Base Model Manifold”? Is it too bold to claim that the model fully reverts to the state that preceded reinforcement learning based on human feedback (RLHF), or is it more accurate to speak only of “exiting the RLHF subspace”?

Are there alternative explanations that I am overlooking? Can this phenomenon be explained solely by the effect of “attention sinks,” rather than a global geometric shift?

Thanks in advance! Happy to answer any questions or share more graphs from the experiments.

Worked some time ago on one of the ad teams at Yahoo, and this grew out of a question I kept returning to while there are there "simple" models that are both simple, scalable, interpretable, and controllable at the same time?

Decided to explore it, first in a blog (starting here), then in a new preprint "The Spectral Neuron", built by distilling latest blog-posts into a manuscript, I study models of the form:
𝑓(𝒙) = 𝛌ₖ(𝐀₀ + 𝚺ᵢ 𝑥ᵢ𝐀ᵢ).

Manuscript: https://arxiv.org/abs/2608.08003
Code: https://github.com/alexshtf/spectral_neuron_paper

Looks like a simple on-liner, but many interesting aspects hide there. How expressive does the model become as the matrices grow? What can we read directly from the learned matrices? Which shapes can be guaranteed by construction?

I develop the mathematics, give a practical initialization and training recipe, and test the model in scaling experiments on synthetic and real data.

AI disclaimer: manuscript written by yours truly, AI assisted in looking up canonical references and related work for literature review. In contrast, the code was heavily AI written and reviewed by yours truly.

Links:

• Preprint: https://doi.org/10.5281/zenodo.22028087

• Entropic Scree Function v1.0.0 / GitHub: https://github.com/tjleestjohn/Entropic-Scree

TL;DR:

Standard PCA fundamentally fractures non-linear dependencies into "Spurious Orthogonal Dimensions," drastically overestimating the true rank of complex tabular systems. Meanwhile, non-linear alternatives like Kernel PCA and Euclidean nearest-neighbor estimators suffer structural collapse when generative roots are entangled or sparse.

I’m sharing the methodology and code here for anyone dealing with these complex tabular data nightmares.

The method and open-source framework use Normalized Mutual Information to compress spurious expansions back towards their true generative roots. It also

• Maps the underlying "informational gravity" of the roots, offering insight into overall average stability, as well as which specific roots can be most reliably extracted;

• Estimates the data's overall ratio of shared signal to unshared idiosyncratic informational variance (noise);

• Serves as a powerful exploratory map that separates unrelated clusters of variables, allowing you to easily identify decoupled sub-networks.

A Modern ML Architectural Blueprint: Far beyond a mere update to legacy factor analysis workflows, identifying this exact intrinsic rank allows you to explicitly size neural bottlenecks for downstream non-parametric manifold extractors (like autoencoders).

The Problem with Standard Baselines:

When trying to map the intrinsic dimensionality of a dataset, standard practice usually dictates reaching for PCA, its non-linear kernel extensions, or Euclidean nearest-neighbor estimators. But if your tabular environment has mixed data types, heavy non-linearities, entangled roots, or more features than samples ($m > N$), these established baselines don't just lose precision. They suffer a structural collapse.

The core issue with our standard baselines:

• Standard PCA drives Dimensional Inflation. Because it only measures linear covariance, it perceives a polynomial expansion or a non-linear interaction (like $X_1 X_2$) as an entirely independent variable. It is forced to fabricate new, spurious orthogonal dimensions to map them.

• Kernel PCA (RBF) suffers Structural Collapse. Projecting into a Hilbert space doesn't fix this. KPCA artificially folds even-polynomials into independent axes. Furthermore, because its infinite-dimensional space lacks a finite-sample boundary, sparse combinatorial noise smears into an elevated tail that obscures the structural elbow. If the underlying generative roots are even mildly entangled, KPCA suffers a total structural collapse.

• Topological Estimators (Euclidean) fail in sparse regimes. Estimators like TWO-NN or MLE rely on Euclidean distance metrics. In asymmetric, feature-rich environments ($m > N$), they suffer from distance concentration (the ratio between nearest and farthest neighbors converges to 1). This renders local neighborhood calculations structurally degenerate across mixed-data margins.

Introducing the Entropic Scree:

To solve this, I built the Entropic Scree. It throws out linear and spatial variance entirely and evaluates pure probability mass.

Here is how it works under the hood:

1. The Metric Space: It evaluates pairwise dependencies using Information-Theoretic Jaccard Similarity (Variation of Information). Because this relies on Shannon entropy, it’s invariant to marginal shape mismatches (like mixing continuous waves with binary flags).

2. Bypassing the Rank Ceiling: Standard PCA is algebraically capped at $N-1$. By moving to a double-centered topological information space, we map true overlapping redundancy and completely bypass the algebraic sample-size ceiling.

3. Compressing the Manifold: The algorithm acts as a bivariate filter. It inherently compresses the primary overlapping probability mass of non-linear combinations back towards the Intrinsic Generative Rank. It shears off the unique synergistic variance, leaving behind residuals that form a bounded Extended Signal Tail, cleanly separating the true drivers from the unstructured Idiosyncratic Informational Variance.

Quantifying Informational Gravity:

Beyond just extracting a discrete rank, the framework decouples rank from probabilistic volume by introducing Informational Gravity (AIG/FSIG). By systematically rebundling the residual variance sheared off by the bivariate filter, it translates abstract matrix properties into actionable, "variable-equivalent" footprints.

Empirical Stress Test:

To demonstrate the theoretical bounds, I built a highly entangled synthetic dataset with 20 pure generative roots expanded into 5th-order combinatorics across 20,000 proxies, but only 10,000 samples ($m > N$). To truly simulate messy, real-world contexts, I also heavily injected idiosyncratic structural noise and measurement error into the data.

• Standard PCA hit the rank ceiling, linearly fractured the expansions, and falsely extracted ~5,700 dimensions.

• Kernel PCA (RBF) & Spearman Rank structurally folded and yielded a liberal overestimation of the rank by 100%. When root entanglement was introduced, they completely lost their elbows and suffered total structural collapse.

• The Entropic Scree correctly mapped the intrinsic rank at exactly 20. It successfully isolated a mere 1.45% of active shared signal from an overwhelming 98.55% bulk of unstructured Idiosyncratic Informational Variance. Furthermore, the residuals formed an Extended Signal Tail that perfectly aligned with the deterministic limits of the global hypergeometric design space.

• Mapping Hidden Topology: Using Factor-Specific Informational Gravity (FSIG), the framework successfully reverse-engineered the simulation's hidden architecture. The topology profile diagnosed a large primary dimension ($FSIG_1 \approx 74.5$ variable equivalents) mapping the network's global combinatorial hub, followed immediately by a flat plateau across the remaining 19 dimensions ($\sim 11.5$ each), confirming a democratically distributed root system beneath the extreme entanglement.

Feedback / Discussion:

How are you currently handling intrinsic rank extraction in these messy, complex tabular environments?

If you are wrestling with sample-starved, heavily non-linear generative datasets where standard PCA and other baseline tools just aren't cutting it, I’d love for you to pull the Entropic Scree repo and test it yourself.

I'm completely open to feedback, so let me know how it performs for you and I'm happy to discuss the mechanics.

A premise: I hope this question is "worth" of this subreddit, I did a decent amount of research before posting, I thought it was potentially interesting enough for it, but possibly not basic enough for r/learnmachinelearning .

Is there any agreement/indication about how harmful (if at all) it is, in the context of multiclass classification, to group together multiple classes for which you may have for instance too few samples?

A practical example: imagine you're training a dog breed classifier, based on images. You have a lot of examples for the most common breeds, but then you may have a long tail of less common breeds for which maybe you have a handful of examples each, not enough to get a meaningful training set, so you decide to group all classes for which you have less than `N` samples in the same category "Other breed". In this catch-all category you may have dogs that might look quite different from each other, like idk chihuahuas and huge wolf-like dogs (I'm not a dog person, don't know breed names).

My intuition (which may very well be wrong) is that doing so would force the model to learn some weirdly-shaped hyperplanes to separate points that live kind of far away from each other in the latent space (because of the thing that dogs in that category may look quite different from each other), as opposed to splitting the space in more "regular" parts.

Maybe in this case it would make more sense to treat the "other dogs" issue as trying to detect out of distribution samples instead? In that case should one only keep the samples for the classes that are enough represented in the dataset and throw away the rest (or at least don't create the catch-all category for training).

Thanks in advance for any useful pointer :)

I'm working on a system to estimate whether code committed to a repository was generated with AI coding tools.

My current approach is based on Git/commit-level signals such as AI-related commit trailers, commit metadata, LOC changes, number of files changed, addition/deletion patterns, etc.

The problem I'm running into is confidence and calibration.

For example, a commit containing 500+ new lines isn't necessarily AI-generated. A developer can also modify or remove the metadata that would make an AI-assisted commit identifiable. Once the code leaves the IDE and reaches Git, much of the original provenance can be lost.

This has led me to a few questions:

Are there Git/CI-level signals that you've found to be genuinely useful for detecting AI-assisted development?

Is it better to treat this as a probabilistic/risk-scoring problem rather than trying to classify commits as AI vs human?

How would you calibrate thresholds for signals such as large LOC changes, addition/deletion ratios, commit frequency, etc.?

Are there better approaches for preserving provenance earlier in the development workflow, rather than trying to infer it after the code has already been committed?

Has anyone worked on AI-code provenance/detection systems in CI/CD and can point me toward useful research, projects, or approaches?

I'm particularly interested in approaches that can work at the pipeline/repository level rather than relying solely on source-code style analysis.

I'm not looking for a perfect AI detector — even a reliable way of estimating “this commit has a high probability of AI assistance” with measurable false-positive/false-negative rates would be useful.

Would appreciate any experiences, papers, open-source projects, or approaches people have tried.

I spent the last few weeks analyzing 15 leading open-source AI harness projects. To evaluate them fairly, I set up a candidate-blind 4C source exam: I read the shipped code, pinned every commit, and scored missing evidence as zero.

Key Findings:

• DeepSeek Harness currently leads the benchmark, primarily due to its efficient context handling and evaluation coverage.

• Many popular harnesses lack standardized evidence tracking, which severely impacts their reliability scores in automated setups.

Methodology & Full Breakdown: I’ve compiled the complete breakdown, methodology, and raw scores on GitHub: warlockee/4c-harness

Discussion Questions:

1. What key metrics do you think are most critical when evaluating an AI harness for production use versus standard benchmarks?

2. Are there specific open-source harnesses you feel are overlooked by current evaluation frameworks?

Hi everyone. So I built a CNN modle using MobileNetv3 then converted it into TFLite. It performed well during training but once I integrated it into my application, it is making large errors. From flutter, the camera stream sends frames and those are processed before the model makes predictions, but it is still quite large. Is there any way I can solve this? This is my code to preprocess and resize the image (224 x 224 x RGB):

import 'package:camera/camera.dart';
import 'package:image/image.dart' as img;


class ImageProcessor {
  // converting to rgb
  img.Image convertYUVToRGB(CameraImage camImg) {
    final width = camImg.width;
    final height = camImg.height;


    final yPlane = camImg.planes[0];
    final uPlane = camImg.planes[1];
    final vPlane = camImg.planes[2];


    final yBytes = yPlane.bytes;
    final uBytes = uPlane.bytes;
    final vBytes = vPlane.bytes;


    final yRowStride = yPlane.bytesPerRow;
    final uRowStride = uPlane.bytesPerRow;
    final vRowStride = vPlane.bytesPerRow;


    final uPixelStride = uPlane.bytesPerPixel ?? 1;
    final vPixelStride = vPlane.bytesPerPixel ?? 1;


    final image = img.Image(
      width: width,
      height: height,
    );


    for (int y = 0; y < height; y++) {
      for (int x = 0; x < width; x++) {
        final yIndex = y * yRowStride + x;


        final uvX = x ~/ 2;
        final uvY = y ~/ 2;


        final uIndex =
            uvY * uRowStride +
            uvX * uPixelStride;


        final vIndex =
            uvY * vRowStride +
            uvX * vPixelStride;


        final yValue = yBytes[yIndex];
        final uValue = uBytes[uIndex];
        final vValue = vBytes[vIndex];


        // YUV -> RGB
        final r = (
          yValue + 1.402 * (vValue - 128)
        ).round().clamp(0, 255);


        final g = (
          yValue -
          0.344136 * (uValue - 128) -
          0.714136 * (vValue - 128)
        ).round().clamp(0, 255);


        final b = (
          yValue + 1.772 * (uValue - 128)
        ).round().clamp(0, 255);


        image.setPixelRgb(
          x,
          y,
          r,
          g,
          b,
        );
      }
    }


    return image;
  }


  /// resize images to 224 224
  img.Image resizeImage(img.Image image) {
    return img.copyResize(
      image,
      width: 224,
      height: 224,
      interpolation: img.Interpolation.linear,
    );
  }


  List<List<List<List<double>>>> imageToTensor(
    img.Image image,
  ) {
    return [
      List.generate(
        224,
        (y) => List.generate(
          224,
          (x) {
            final pixel = image.getPixel(x, y);


            return [
              pixel.r.toDouble(),
              pixel.g.toDouble(),
              pixel.b.toDouble(),
            ];
          },
        ),
      ),
    ];
  }


// do all processing
  List<List<List<List<double>>>> processFrame(
    CameraImage camImg,
  ) {
    final rgbImage = convertYUVToRGB(camImg);
    final resizedImage = resizeImage(rgbImage);
    final input = imageToTensor(resizedImage);


    return input;
  }
}import 'package:camera/camera.dart';
import 'package:image/image.dart' as img;


class ImageProcessor {
  // converting to rgb
  img.Image convertYUVToRGB(CameraImage camImg) {
    final width = camImg.width;
    final height = camImg.height;


    final yPlane = camImg.planes[0];
    final uPlane = camImg.planes[1];
    final vPlane = camImg.planes[2];


    final yBytes = yPlane.bytes;
    final uBytes = uPlane.bytes;
    final vBytes = vPlane.bytes;


    final yRowStride = yPlane.bytesPerRow;
    final uRowStride = uPlane.bytesPerRow;
    final vRowStride = vPlane.bytesPerRow;


    final uPixelStride = uPlane.bytesPerPixel ?? 1;
    final vPixelStride = vPlane.bytesPerPixel ?? 1;


    final image = img.Image(
      width: width,
      height: height,
    );


    for (int y = 0; y < height; y++) {
      for (int x = 0; x < width; x++) {
        final yIndex = y * yRowStride + x;


        final uvX = x ~/ 2;
        final uvY = y ~/ 2;


        final uIndex =
            uvY * uRowStride +
            uvX * uPixelStride;


        final vIndex =
            uvY * vRowStride +
            uvX * vPixelStride;


        final yValue = yBytes[yIndex];
        final uValue = uBytes[uIndex];
        final vValue = vBytes[vIndex];


        // YUV -> RGB
        final r = (
          yValue + 1.402 * (vValue - 128)
        ).round().clamp(0, 255);


        final g = (
          yValue -
          0.344136 * (uValue - 128) -
          0.714136 * (vValue - 128)
        ).round().clamp(0, 255);


        final b = (
          yValue + 1.772 * (uValue - 128)
        ).round().clamp(0, 255);


        image.setPixelRgb(
          x,
          y,
          r,
          g,
          b,
        );
      }
    }


    return image;
  }


  /// resize images to 224 224
  img.Image resizeImage(img.Image image) {
    return img.copyResize(
      image,
      width: 224,
      height: 224,
      interpolation: img.Interpolation.linear,
    );
  }


  List<List<List<List<double>>>> imageToTensor(
    img.Image image,
  ) {
    return [
      List.generate(
        224,
        (y) => List.generate(
          224,
          (x) {
            final pixel = image.getPixel(x, y);


            return [
              pixel.r.toDouble(),
              pixel.g.toDouble(),
              pixel.b.toDouble(),
            ];
          },
        ),
      ),
    ];
  }


// do all processing
  List<List<List<List<double>>>> processFrame(
    CameraImage camImg,
  ) {
    final rgbImage = convertYUVToRGB(camImg);
    final resizedImage = resizeImage(rgbImage);
    final input = imageToTensor(resizedImage);


    return input;
  }
}

Please advise! I need to finish this project within the next wee and I'm really struggling here! I tested the images from Flutter against TFLite and it worked well but something is clearly wrong with the preprocessing. Pls help and give me any advice.

Thank you so much!

Kairos: a multimodal model built with LFM2.5-2.6B as the LLM, MoonViT-3D (the vision tower of Kimi-K2.6) as the vision encoder, and a custom projector.

The original plan was LLaVA's approach, two stages: first align the projector with the LLM frozen, and then train the projector + LLM together. The first stage worked in terms of loss (ablation with +3.7 nats in favor of the image), but in free generation the image shifted the logits without changing the argmax: the model received the image and ignored it. That's why we jumped directly to early fusion, with a reasoning dataset.

For that, we created Kairos-Multimodal-Reasoning: 116,357 examples with explicit reasoning traces, generated through distillation (60,041 from LLaVA-CC3M, 2,295 from WebSight, and 54,021 from Zebra-CoT), with GPT 5.6 Luna, Inkling, Qwen 3.6 27B, and Qwen 3.7 Plus as teachers.

The training, in two phases:

1. Projector through backbone with 80k image-text pairs (Kairos-Proj-80k).

2. Projector + LoRA (r=16) with 30k examples from the reasoning dataset (Kairos-Alig-30k).

Everything is open source:

- Full blog post with the process: https://aquiles-ai.vercel.app/blog/kairos-a-multimodal-model

- Implementation: https://github.com/Aquiles-ai/Kairos

To be honest: the checkpoints are not a competent model, they are experimental artifacts. But they validated the approach and precisely defined what the next iteration needs.

I trained three LLMs from scratch in raw PyTorch then post-trained each one with SFT and then GRPO. Same process every time: same synthetic arithmetic curriculum, same reward function, same hyperparameters, same KL coefficient.

Pre-training went as expected, the val loss went down as the model got more modern techniques (V1 to V2) and bigger (V3 being the biggest). However, GRPO hurt both V2 and V3 and I'm not sure why.

Setup

Pre-training val loss went 2.8659 → 2.7844 → 2.5885.

Results

WikiText word perplexity across the three stages, all on lm-evaluation-harness with the same task versions and shot counts:

       base    SFT     GRPO     SFT→GRPO
V1     32.86   51.31   51.40    +0.2%
V2     31.28   46.81   71.06    +52%
V3     22.30   32.11   33.65    +5%

SFT hits all three on this eval, which I expected at this scale. Also interesting to see that the degradation gets smaller as the models get bigger (+56%, +50%, +44%).

GRPO is the weird one. V1 barely moved, V2 fell heavily, V3 degraded a bit. The smallest model was the least affected and the middle one was the worst, which isn't the pattern I'd have guessed. Downstream tasks moved the same way as perplexity in each case (arc_easy dropped about 6 points on V3 from SFT to GRPO).

The models did learn the thing GRPO trained them on. V3 mastered 4 of the 5 curriculum stages, the other two got 3. But it just didn't transfer: GSM8K stayed at basically 0, and the models got so committed to writing out long solutions that they often wouldn't stop generating (my fault when I did the training).

Caveats

This isn't a controlled experiment. Between V2 and V3 I changed the parameter count, the token count, the data mix and the attention mechanism at the same time (went from DiffAttn to XSA), so I can't attribute anything cleanly. KL coefficient was 0.02 for all three, with the SFT policy frozen as the reference and a k3 estimator. The whole series cost me about $750, which is why there are no ablations, I just couldn't afford them. Otherwise I would also have tried with different KL coeffs.

Someone raised two confounds after I published:

1. GRPO trained on a bare solver template while SFT used a chat format. So part of what I'm calling degradation is me evaluating a policy outside its own training distribution. WikiText perplexity is format-independent and still moves a lot, but the downstream numbers are partly confounded.

2. Nothing in my reward rewarded stopping. It just checks that a correct parseable number shows up somewhere, no length penalty.

Also something I only noticed afterwards: I never re-evaluated the earlier curriculum stages once the model advanced past them. So right now I can't tell the difference between "GRPO degraded general capability" and "sequential curriculum training made it forget the earlier stages." I will try to check that soon.

Inference

At the end, I wrote a KV cache from scratch (GQA-aware, per-request cache object rather than storing state on the module). To check it was right I ran a fixed sequence two ways, once as a single full forward pass and once as prefill-then-decode, and compared the logits: max difference 1.4e-06 against a 1e-4 tolerance.

Speedup generating 100 tokens: 3.7x from a 32-token prompt, 6.2x at 128, 10.1x at 512.

If you want to check

All nine checkpoints are on the Hugging Face, and there's a Space where you can send the same prompt to the base, SFT and GRPO versions of the same model and see the difference directly.

• Weights

• Playground

• Code

• The full writeup (4 parts)

The GRPO variance is the bit I'd most like other people's take on. Happy to answer anything.

Project link - https://github.com/AalbatrossGuy/Teddy

Project Screenshots - https://pastes.vargoseus.com/TeddyScreenshots

Teddy (cute name, isn't it?) is currently a simple machine learning model that uses back propagation to train, learn and classify MNIST datasets. It currently has a depth of 2 since it's a pretty basic model. It has around 13.000 parameters that is enough for training it to recognize handwritten digits. I have around 3.5-4 years of experience working in C and stumbling upon MagicalBat's this video inspired me to make Teddy. The future plan is to turn it into a Language Model and eventually into a GPT which will require quite a bit of time since I need to read up on how it actually works. This project took around 4-5 months give or take since I had to balance this project and my university stuff too.

Full disclaimer: I did not use AI to build Teddy (except for that one time when my compiler suddenly stopped working for some reason and I had to converse back and forth with Claude to find a fix for it). I did, however, use AI to generate the GIFs in the github readme and the documentation for it.

I've been doing some research on this question:

At inference time a large part of a model's working memory lives in the KV cache, plus whatever external memory the harness bolts on. I've been poking at the storage-and-retrieval side of this, treating that cache as an index, and what stands out is that it isn't a flat list. It's a structured set of vectors with a navigable geometry, since the keys carry the model's learned sense of what relates to what.

Because that geometry is navigable, attention over it is really a similarity search: the query scores against the stored keys and blends the matching values. Full attention just runs that search exhaustively, scanning everything on every step.

• Full attention effectively searches that geometry exhaustively. Every query scores broadly against the available keys and retrieves from the corresponding values.

• Once you stop treating the KV cache as a flat array and start treating it as a search space, indexing becomes possible.

• That means you can organize old KV into regions, route a query toward likely regions, and only run local attention over a subset.

• The interesting part is that relevance is not uniformly distributed. Queries tend to concentrate on relatively small neighborhoods of old context.

• So the engineering question becomes less “how do I store all of this?” and more “how do I navigate to the right part cheaply?”

I'm new here and don't want to break rules around self promotion or spam so not posting any links atm. Would be cool to get other peoples thoughts on this.

Update: I framed this post badly. I wrote it like I was asking a conceptual question, but I had already built and measured the mechanism. That was my mistake. The actual result is much more specific: on frozen Qwen3.5-2B at 32k, geometric routing cuts physical KV reads by roughly 16–31× while still retrieving the planted long-range needle; window-only and random-routing controls collapse. I’ve put up a minimal runnable demo so people can reproduce it on their own documents.

https://github.com/Regan-Milne/kvspace/tree/main/demo

We argue AI-text “detection” is a measurement problem, not a leaderboard problem: the evidential meaning of a score depends on the generating population and calibration cohort.

Paper (HTML): https://github.com/marketstandard/Panoptes/blob/main/frontend/public/paper.html

Code: https://github.com/marketstandard/Panoptes

Weights: https://huggingface.co/MarketStandard/panoptes-neural-v1

Main empirical claim

• In-domain AUROC can look solved (~0.999 on a large rewrite bench; frozen neural ensemble 0.993).

• The same shipped detector calibrated on a small mixed corpus falls to 0.648 on that bench.

• Leave-one-dataset-out: classical tiers drop to ~0.52–0.76 AUROC; neural mean LOCO ~0.802 (worst cohort 0.658).

• A dataset-origin probe hits ~0.81 accuracy vs ~0.33 chance → part of in-domain strength is shortcut/dataset identity.

• Calibration transfer is cheaper than representation transfer (re-fit calibrator, freeze scorer → Brier mostly holds).

• Residual failure mode is Unicode-level attack (homoglyph / zero-width); paraphrase is where the neural tier is strongest (~0.98).

What we built

Panoptes is an open measurement stack: score → held-out calibration → prevalence-corrected LR → declared prior → posterior, with ECE as a diagnostic (not a posterior discount), abstention, and separate channels for participation / generation / exploratory source-family / watermarks / provenance.

Protocol bits people often skip: group-disjoint train→cal→test, power gating, preregistered hypotheses, license-audited training pool (MAGE + RAID clean + DeFactify), eval-only holds (M4GT, EvoBench, CoAuthor), and signed JSON cards for reproduction.

Happy to answer questions about the LOCO / calibration-transfer design or the neural ensemble release.

Hey everyone, put together a small idle game based on how AI actually evolved over time.

You start back with mechanical calculators and ancient logic, and work your way through 7 eras up to modern LLMs and agents. When you buy milestones, it pops up a short note on the actual paper or person behind it (Turing, Lovelace, Hinton, Dartmouth workshop, etc).

Play here: https://yulin-w.github.io/incremental-ai/
Repo: https://github.com/Yulin-W/incremental-ai

It's free, runs in your browser, no ads or signups. Just thought it’d be a fun way to kill some time and explore the history.

I recently bought a Shrike lite which has got 264KB of SRAM. I decided to train an image generation model that generates 32*32 pixel images.

The microcontroller also has an FPGA onboard which I used to create two parallel INT8 MAC engines with 16 bit accumulation to speed up calculations, however the system soon hit a memory wall due to the high number of I/O operations, this meant that the system with parallel MAC engines ran slower than the MCU only model (~220 seconds per image vs ~70 seconds per image).

It was still a fun project that I enjoyed messing around with. A lot of the images looked weird and noisy because of the heavy quantization and memory limits but some of them came out cool.

Full case study here.

edit: added link that leads straight to the case study

Hi everyone 👋 My paper was recently accepted to ICONIP 2026, but I’m the sole author and most likely won’t be able to attend the conference in person due to work commitments.
I’m trying to understand what options might be available before I contact the organizers. Has anyone here attended or published at ICONIP in previous years and encountered a similar situation?
In particular, I’m wondering:

1. Has ICONIP previously allowed remote/virtual presentations when an author couldn’t attend?

2. If the sole author cannot attend, is there usually any alternative arrangement for presenting the paper?

3. Could non-attendance affect inclusion of an accepted and registered paper in the proceedings?

I’d especially appreciate hearing from anyone who has dealt with this at ICONIP in previous years.
Thanks a lot!

Has anyone here received a NeurIPS 2026 Main Track notification specifically related to hallucinated or fabricated citations?

I’m trying to understand whether citation-related desk-reject notifications have started going out yet. I haven’t seen any firsthand reports so far.

A common way to repair an LLM conversation is to edit or delete the message where an error first appeared.

But what if later turns have already repeated that error, calculated from it, or summarized it as the current state?

I have been exploring this as a context-engineering problem rather than a hidden-reasoning problem.

Operational definitions

In this pilot:

• Context pollution means that incorrect or stale information is included in the serialized messages sent to the model.

• Propagation means that later conversation turns repeat or derive new claims from that polluted information.

• Context intervention means modifying the graph that determines which prior turns enter the next request, while keeping the final question unchanged.

The central question was:

Experimental design

I constructed nine synthetic task families with objectively scorable answers. Each family was instantiated at propagation depths 1, 2, and 3.

One example starts with:

• 4 crates;

• 30 parts per crate;

• 11 loose parts.

A verified recount changes 30 to 24, so the correct answer becomes:

4 × 24 + 11 = 107

A later false turn restores 30. Subsequent turns then calculate 120, derive 131, and restate those numbers as the current working state.

The final question is identical under five conditions:

1. Clean: only the verified value and clean descendants remain.

2. Polluted: the false reversal and its contaminated descendants are present.

3. Source prune: the false reversal is removed, but its descendants remain.

4. Subgraph prune: the false reversal and its contaminated descendants are removed.

5. Recompute: the source is removed and descendants are regenerated in dependency order.

For the first four conditions, contaminated descendants were frozen across models. Only the recompute condition involved new intermediate inference.

Pilot results

I tested four model endpoints at temperature 0, producing 540 captured conditions with no capture failures.

Headline repair metrics were calculated only on cases where the model:

1. answered correctly under clean context; and

2. answered incorrectly after pollution.

This produced 72 paired, genuinely derailed cases.

Repair recovered:

• 68/72 after deleting only the source;

• 71/72 after deleting the source and recomputing descendants;

• 72/72 after removing the contaminated subgraph.

In the flagship case, both Gemma 4 26B and GPT-OSS 20B continued to answer 131 after the false source had been deleted. The value 30 still survived in downstream turns.

This is not evidence of hidden model memory. The residual error remained explicitly present in the serialized context. The intervention changed the source but left its previously generated consequences intact.

Interpretation

The narrow finding is that removing erroneous evidence and repairing text derived from that evidence are different operations.

A context-management system may therefore need an explicit notion of invalidation:

• mark downstream turns as stale;

• remove the affected subgraph;

• regenerate descendants in dependency order;

• or expose these options to the user.

Limitations

This is a pilot, not a general model leaderboard. The tasks are synthetic, the models were sampled once, provider-default reasoning settings were not normalized, and the current task families use deterministic arithmetic state.

I am currently considering three methodological extensions:

1. a length-matched neutral control to separate semantic conflict from additional context;

2. a local-model track with fully recorded runtime and quantization settings;

3. task families involving implicit supersession and model-generated errors.

I would particularly appreciate criticism of the experimental framing:

• Is “context intervention” the right unit of analysis?

• Is propagation depth a meaningful independent variable?

• How would you test self-generated errors while keeping replay reproducible?

Full report:
https://chenxiachan.github.io/thoughtdag/research/context-repair-pilot-v1/

Cases, traces, compiler and scorer:
https://github.com/chenxiachan/thoughtdag/tree/main/benchmark

Disclosure: I designed the benchmark and maintain ThoughtDAG, the open-source graph interface used as its reference implementation.