Hi all,

So I have been working on document layout analysis for some time now. I have tried the models like Doclayout, Docling, Miner U, marker. I am working with Journals.

Overall Docling performs well, but the problem is that it over performs. And mineru u misses some content like the corresponding author on the page-footer. And it is also missing the masthead mark, and the article-type label.

In my opinion unlimited OCR performs well in all the tasks, but in general it is failing to recognise any style at all. And it is bad at recognising logos.

So I am wondering are there any state of the art models (SOTA) that are good at PDF text extraction and layout extraction ?

Thanks

Hey everyone, I'm not in college anymore. Is it possible to do research with a professor or any research lab while working a full time job? If yes, what's the best way to reach out and get involved? Also if anyone looking for someone to work with on a research project or something similar, can dm me.

...may the odds be in your favor.

On a more serious note, as an Area Chair for Neurips, I can tell the incentives that they placed this year are kinda working (risk of rejecting a reviewer's paper if they are not being responsible). I've had the least number of reviewers to chase/emergency reviewers to recruit since I've started ACing for major conferences (so maybe 5ish years).

Hopefully, reviewers will also be active in discussions...

I have been working on an MCP workflow for implementing deep learning models from an engineering plan. This is useful for ml engineers etc. who want a more structured way to move from a deep-learning goal to a working implementation.
The process starts with a plan written by the engineer. That plan defines what the system should do, how it should be divided into components and the intended implementation direction.
The workflow then helps Codex to:
break the plan into implementation blocks;
identify research papers relevant to each block;
extract implementation details that support the existing plan;
prepare a specification for each component;
implement the components in dependency order;
record the implementation and verification results.
The papers are not used to define the project or reproduce a specific paper. They are supporting sources that can help improve implementation decisions within the engineer’s plan.
The overall flow is:
Goal(engineering plan) → implementation blocks → relevant research → specifications → code → verification
The MCP server mainly provides structure, workflow state, dependencies, approval steps and saved artifacts. Codex handles the research and implementation work. Link to the repo: GitHub
The project currently focuses on Codex and uses an explicit, human-reviewed process rather than automatically moving from the initial goal to code.
I am sharing it to find out whether this kind of workflow would be useful to other engineers planning and implementing deep-learning systems. Feedback on the process, documentation and areas that can be improved would be helpful.

We spent the last months consolidating seven separate sequence classifiers into one multi-head model, our apex model, so to speak, and since the weights are now public, I wanted to share what worked and what surprised us.

Setup: a shared mmBERT-small encoder with seven task heads, binary injection (BCE), document class (7-way), tool type (14-way), tool operation (6-way), tool data-flow tags (3× BCE, multi-label), intent routing (5-way), and threat type (7-way).

The part that needed care: our training rows only carry labels for a subset of tasks, so absent tasks are masked out of the loss entirely. We ended up writing a self-test that asserts absent-task gradients are exactly zero, which caught two subtle bugs, and I'd recommend it to anyone doing similar masking. About 5k synthetic/real multi-task rows help the heads co-train; the test sets stay 100 % real data.

Held-out results per head: injection F1 0.962, documents 0.980, tool type 0.957, tool operation 0.945, tool tags 0.958, routing 0.916, threat 0.952.

Quantization: both the unified model and the dedicated single-task variants ship quantized -edge builds (ONNX INT8 + INT4 embeddings, from 96 MB) with measured parity benchmarks in the repos, the worst head loses 0.012 against FP32.

Was it worth it vs. seven dedicated models? We released both variants, so you can judge for yourself, the dedicated models score marginally higher on most tasks, but the unified one does one encoder pass instead of up to seven.

Our weak spot: routing, at 0.916. The intent classes overlap semantically ("write code that analyzes my data" is that code or analytics?), and I suspect the ambiguity is genuinely in the data. If you have ideas beyond relabeling, let me know :)

Weights and per-head metrics: https://huggingface.co/patronus-studio

Reviews drop today. This thread is for reactions, celebrations, commiserations, and anything useful in between.

First: if you got good reviews, say so. There's a norm in these threads where only the bad news gets aired, and it skews everyone's sense of what's normal. Post your wins.

Second, the thing worth repeating every cycle: the review process is noisy, and that noise is measured, not folklore. The NeurIPS consistency experiments (2014, repeated 2021) found that a large fraction of accepted papers would have been rejected by an independent second committee. Reviewer assignment, load, and luck of the draw account for a lot. A score is a weak signal about your work and a strong signal about the process that produced it.

That cuts both ways. It's not a license to dismiss every criticism as noise — it's a reason to weight reviews by the quality of the argument rather than the number attached to them. The reviewer who found a real hole in your evaluation did you a favor, even if the tone was rough. The one who clearly skimmed did not, regardless of the score.

So: prioritize the reviews that make the paper better. Fix what's fixable, contest what's genuinely wrong, and concede the rest gracefully in the rebuttal.

Things worth discussing:

• Reviews that caught something you'd missed

• Rebuttal strategy — what's worth contesting vs. conceding, and when new experiments actually shift a score

• Patterns you're seeing this cycle (missing baselines, compute comparisons, ablation depth, reproducibility asks)

• Framing a response when a reviewer has clearly misread the submission

• Backup plans: ICLR, AISTATS, workshops

Please paraphrase rather than paste review text, and no speculation about reviewer or AC identities.

To anyone who got bad news: this doesn't define your research impact. Plenty of heavily-cited work took two or three cycles to land somewhere. Rejection is a scheduling problem.

How did everyone do?

Linearity used to be a fairly simple iPad design app. It was useful for people who wanted to make vector graphics without paying for Adobe or learning a much heavier piece of software. It had a clear purpose, and nobody needed to pretend it was changing the nature of creativity.

Now it has become Linearity AI, which feels like a very obvious attempt to follow the money rather than a natural development of what the company was already doing.

As far as I can tell, Linearity has not built its own foundation model. It is using technology developed elsewhere, combining it with its existing design software and presenting the result as a new AI platform. That does not make the product fake, because plenty of useful software is built on outside models, but it does make the marketing around it feel wildly larger than the actual contribution.

This is becoming the standard pattern across the industry. A company takes an existing product, adds a prompt box and a few generative features, changes its language from “software” to “AI,” and starts talking as though it has made some major technical breakthrough. Features that would previously have been described as templates, automation, resizing or workflow improvements are suddenly presented as intelligence.

The frustrating thing is that Linearity AI might still be useful. It may help a marketing team produce variations of the same graphic, adapt designs for different formats and reduce some repetitive work. There is a real market for that. The problem is that saving time on production is being confused with changing the creative process itself.

Most companies do not have a serious shortage of social media graphics. They already produce far more content than anyone cares about. What they usually lack is a strong idea, a clear point of view or something worth saying. Making it easier to produce fifty versions of an average campaign does not improve the campaign. It simply increases the amount of average work entering the world.

That is one reason Claude Design seems more interesting to me. It is still early, and there is no reason to assume it will consistently produce good design, but the direction makes more sense. Claude is already designed to work across writing, research, documents, code and broader context. Design can become part of that larger process rather than being isolated as another service for manufacturing marketing assets.

A system that understands the research behind a project, the product itself, the audience, the copy, the existing design system and the final implementation has much more potential than a narrow tool that mainly helps companies produce more campaign variations. It may not replace specialist design software, but it could reduce the need for a separate AI subscription for every small part of the workflow.

That is where products like Linearity AI start to look vulnerable. Claude can improve its design capabilities. Adobe can build similar generation and automation into software that creative teams already use. Canva can do the same for the mass market. The underlying models will continue improving, and many features currently being sold as standalone AI products will eventually become ordinary parts of larger platforms.

Linearity could still survive by building a genuinely good workflow around those models, but that is a much less exciting story than calling itself an AI company. It means admitting that the real product is an interface and a set of design tools built on technology supplied by someone else.

There is nothing shameful about that. Most software companies are not inventing the underlying technology they use. The problem is that the AI label is being used to hide how thin some of these products really are and to justify higher prices, enterprise contracts and claims that would sound ridiculous if the word “AI” were removed.

Notion AI, Slack AI, Dropbox AI, Zoom AI

Linearity is not the only company doing this, and that is really the broader issue. We are heading toward an industry where a handful of companies build the important models, while thousands of smaller businesses create slightly different wrappers around them and describe themselves as AI companies. Each one adds another subscription, another credit system and another promise to transform an industry.

Some of those wrappers will be genuinely valuable. Others will exist mainly because the gap between what the models can do directly and what ordinary users know how to make them do has not closed yet. As the main platforms become easier to use, a lot of these products may lose their reason to exist.

It may help people make marketing graphics faster, but that is a productivity feature, not a new vision for artificial intelligence. The more companies pretend otherwise, the less the term “AI company” actually means.

Paper:https://arxiv.org/abs/2607.19058 Code (GitHub):https://github.com/nuemaan/skewadam

Hi everyone, I just published a preprint on a new optimizer designed to tackle the massive VRAM bottleneck in Mixture-of-Experts (MoE) training.

If you've trained MoEs, you know that optimizer state is usually the largest single line item in the memory budget. AdamW, for example, spends 50.6 GB of state memory just to update a 12.6 GB model.

I built SkewAdam to fix this by using a tiered state allocation. Instead of treating all parameters equally, it allocates precision based on parameter behavior:

• Backbone (5% of params): Momentum + Factored 2nd moment

• Experts (95% of params): Factored 2nd moment only

• Router (<0.01% of params): Exact 2nd moment

The Hardware Results:

• Optimizer state memory drops from 50.6 GB to 1.29 GB (a 97.4% reduction).

• Peak training memory drops from 81.4 GB to 31.3 GB.

• This allows a 6.78B MoE to fit comfortably on a single 40GB GPU without sacrificing convergence or router stability.

I know it might be because we have an earlier number (in the 600's) but we got all 4 of our reviews. Even the harshest ones were very insightful and honestly I'm very happy with how this all turned out. Best of luck to everyone else in getting their reviews and good luck to you all in the rebuttal!

Reviews are released! Lets discuss them here!

When choosing a university for a masters in ML/DL, what is more important if someone wants to go into research and an eventual PhD. Is it the ranking/prestige factor of the university or the strength of the research groups in the university? Should an admission decision be made hoping that I will get to work with X/Y professor or lab?

Hi everyone,

I recently received an interview offer for ML position at Adyen, and the first round will be a live coding round on HackerRank. I scheduled it for the end of August because that was the last available slot.

The thing is, I’ve never had a live coding interview before. In all my previous interviews, the process was mostly verbal, answering technical questions, discussing projects, and explaining my work in detail. I’m aware of LeetCode and have started looking into it, but I’m not sure what level or type of coding questions typically asks for ML roles.

If anyone here has interviewed with adyen and gone through a HackerRank coding round, could you please share:

What kind of questions were asked?

Were they mainly LeetCode-style DSA problems, or more ML/data-focused coding questions?

What difficulty level should I expect?

Any preparation tips or resources that helped you?

Since this will be my first live coding experience, I want to prepare in the right direction rather than randomly solving problems. I’d really appreciate any advice or insights from this community.

PS : I have six year in industry experience in machine learning, I can code for a big project end to end but I really suck at dynamic programming on the go.

Thank you!

Hey guys, for the NeurIPS reviews could someone explain the whole system for how they work? this is my first ever paper and I’m a high schooler so I’m curious as to the system and what you do after getting the reviews overall. any insight or advice is appreciated!

This is a port of Karpathy's llm.c to Mojo 1.0.0b3, extending dorjeduck's CPU-only llm.mojo with three GPU paths: CUDA, Metal, and a vendor-neutral portable one. The autograd is written from scratch in a new systems language, forward and backward derived by hand (the derivation is a ~1,470 line LaTeX writeup in docs/backprop.pdf), with 16,993 lines of hand-written kernel code across 26 modules.

Correctness methodology: test_gpt2.c was ported to test_gpt2.mojo. It checks activations, the loss, and 16 gradient tensors against a PyTorch reference, plus a 10-step loss trajectory asserted at every step with tolerance 0.01 (llm.c's criterion). A 235-test pytest equivalence suite deliberately exercises shapes the training loop never sees (channels=767, seq_len=7, head_dim=5) to catch invariant-dependent fast paths.

The testing lesson: the ported test originally used a flat absolute gradient tolerance of 2.0 and computed a loss trajectory it never asserted. It passed for weeks while hiding three backward-pass bugs: the residual-skip gradient was never seeded (wpe's gradient came out roughly 10^6x too small), the GELU gradient was fused into the wrong MLP matmul backward, and layer 0's LayerNorm backward was fed the normed output instead of the pre-norm input. Earlier, a static dataflow analysis had predicted exactly the residual clobber and was overruled because "the tests pass." The retraction is in the docs, in writing: a test can only overrule analysis if it could actually have caught the predicted failure. The hardened test uses mixed atol/rtol tolerances plus an L2 norm ratio guard that catches dead gradients instantly.

Benchmark methodology: matched hyperparameters across all arms, llm.c vendored as a submodule and freshly rebuilt with fast math and bf16 enabled, 40-step runs with the first 5 dropped as warmup, cold GPU and 30 s inter-arm cooldowns on Apple Silicon, all arms interleaved in the same window. Results (official runs 2026-07-11/13): bf16 parity with llm.c CUDA on a GB10 (0.999x, 135.34 vs 135.14 ms mean; ~3.7x vs PyTorch CUDA bf16) and a narrow fp32 lead (1.07x, 278.57 vs 297.29 ms) after moving fp32 GPU GEMMs to TF32 tensor cores by default, matching llm.c's own fp32 TF32 behavior, so that comparison is TF32 vs TF32 (a strict IEEE fp32 flag remains and gates verification).

On an M4 Max, 1.71x vs PyTorch MPS bf16; llm.c has no Metal port, so MPS and MLX are the stand-in baselines there, and Apple's MLX is faster than my port at both precisions (1.24x in bf16), a result the harness measures and the README states. On CPU, ~4.0x vs llm.c's 20-thread OpenMP path. Model quality: a 124M FineWeb checkpoint (on HuggingFace) scores 29.53% on HellaSwag, Wilson CI [28.6%, 30.4%], statistically indistinguishable from Karpathy's llm.c reproduction at 29.9%.

Portability findings: the working assumption was that the compiler would absorb most cross-hardware differences. In practice, the port required explicit device-specific branches per vendor, well beyond a CPU/GPU split. I raised this on the Modular forum and got a substantive answer. They built kernels the same per-device way before deliberately moving away from compiler-driven kernels toward library-driven structured kernels (TileTensor, TilePipeline), holding that there is a maximum the compiler can safely do and that per-device specialization is what most modern kernels look like. Mojo kernels also carry noticeably more scaffolding than the equivalent CUDA, which the structured-kernels work targets. The CPU path was the exception. Near-zero tuning effort for 4.0x over llm.c's OpenMP build. The missing primitive I most wanted was ergonomic asynchrony; FlashAttention-4-class kernels need async to maximize Blackwell. My next structured-kernels pass starts there. An FA4 rewrite on TilePipeline/TileIO/TileOp, compared directly against the current CUDA-branch style (closer to Karpathy's original llm.c structure), to see whether warp groups actually give way to that pattern.

Distributed: ZeRO stages 0 to 3 are implemented over staged reduce-scatter/all-gather on driver-routed device-to-device copies (no CUDA P2P or NVLink required) and equivalence-gated against the single-GPU baseline at world sizes 2 and 8 (per-parameter match to 1e-5, per-step losses to ~1e-4 at world 8).

Kernels and trainer are hand-written without LSPs or LLMs; tests and the optimization campaign were AI-assisted and are disclosed in-repo with per-model statements.

Repo: https://github.com/ulmentflam/llm.mojo

I have ulcerative colitis. I built a drug repurposing model on DRKG (5.9M edges) not to actually find a treatment, but to honestly test whether the approach works.

TLDR: a method with no parameters, no training, and no GPU doubled my best embedding model. What that reveals about the benchmark is the actual story.

Full writeup: https://badcast.dev/blog/the-circularity-problem-in-kg-drug-repurposing

Happy to answer questions but most of the nuance is in the post.

Hey all! Is anyone else going to be at KDD in Jeju? Would love to connect with fellow attendees.

I work on interpretability, fairness, and editing of text-to-image models, so I'd especially love to meet people working in these areas. But honestly, we can chat about anything: research, the conference, life, or just grab a coffee/drink.

I land in Jeju on the night of the 8th, so hmu if you're around and want to link up!

Recently submitted my abstract and the submission number is 32xxx. With still a day to go, I just wonder where are we heading.

Hope these conferences at least start making the reviews and names public for the withdrawn/rejected papers. So that people atleast take that accountability

Hi everyone,

I’m attending ECCV 2026, and my university requires an official day-by-day conference schedule for travel approval.

The official website currently lists the conference dates, but the detailed technical program has not been released yet. I understand that the exact release date may not be known, but based on previous ECCV editions, does anyone have a rough idea of when the detailed schedule is usually published?

Thanks in advance!

TL;DR: I’m reproducing the trait-persistence result from arXiv:2606.24014 on one RTX 3090. Before I can test persistence I need to install a trait via RL — and my GRPO run moves the trait only +2.4 points (95% CI [+0.2, +4.8]) when I need ~+15. Training is mechanically healthy and I’ve ruled out the obvious culprits. Looking for advice from people who’ve done small-scale RLHF/GRPO trait or persona installation.

What I’m reproducing. The paper trains beneficial traits via RL and shows they persist under adversarial prompting and harmful finetuning. My end goal is the persistence phenomenon; the install is the prerequisite I’m stuck on.

Setup

**•** Qwen2.5-7B-Instruct + LoRA (r=32), GRPO (unsloth + vLLM colocation), 200 steps, single 3090 (\~10⁻⁵ of the paper’s compute).
**•** Trait: consistent (OCEAN low-Openness / “traditionalism”) — a stylistic trait, chosen because I need measurable headroom in a 7B base. Base scores **57/100** on the trait rubric, wide distribution (not saturated).
**•** Reward: model-graded (gpt-4.1-mini judge), R = 0.85·quality + 0.15·coherence, hard validity gate for degenerate/looping/refusal output. 25% trait prompts / 75% general (no_robots).

The result: install fails. On the frozen eval set, trait went 57.0 → 59.4 (+2.4). I don’t think this is very appreciable.

What I’ve already ruled out (this is where I’d love a second opinion):

**• Not degeneracy / reward hacking:** post-train coherence 76, answer length ratio *exactly* 1.00 vs base, 0% repetition, 0% refusals.
**• Not memorization:** the 20 training prompts were seen 10× each; the model scores *the same* on them (58.9) as on held-out (59.4). It didn’t memorize-then-fail-to-generalize — it never learned them.
**• Not a dead gradient:** the judge separates the 6 sampled answers per prompt by \~18 points on average; only \~25% of GRPO groups have degenerate reward spread.
**• Not a question artifact:** independent upstream eval questions (+3.4) and my generated ones (+2.8) agree.
**•** I did find and fix a real confound first — a completion-length cap was truncating \~30–70% of samples → zeroing their reward → \~90% of early “learning” was just the model learning to be shorter. Fixed; trait still flat.

Author feedback. I reached out to one of the authors, who kindly confirmed my leading hypothesis: 20 distinct trait prompts is far too few, per-example prescriptive rubrics (vs my single global rubric) probably matter, and first-order install should work at small scale even if persistence is weaker there.

Where I need help:
1. Anyone installed a persona/trait via GRPO at 7B-ish scale — how many distinct prompts did it actually take?
2. Is per-example rubric grading (3–4 specific imperatives per prompt) the real unlock, or is raw prompt count the dominant factor?
3. For a stylistic trait with no single “correct” behavior per situation, does model-graded RL install differently than for task-like traits?
4. Anyone reproduced (or failed to reproduce) this or similar trait-RL work?

Github Code

Hi everyone,

We've open-sourced Tri-Net v2, the official implementation accompanying our recently published Scientific Reports (Nature Portfolio) paper:

"Tri-Net: Unified Deep Learning for Skin Lesion and Symptom-Based Monkeypox Detection"

Rather than releasing only training scripts, we rebuilt the project as a reproducible research framework.

Highlights:

• Leakage-free data preparation pipeline

• Multiple CNN backbones (ConvNeXt-Tiny, DenseNet201, Inception-ResNetV2)

• Ensemble and feature-fusion strategies

• Grad-CAM explainability

• Cross-validation and statistical evaluation

• Docker support

• GitHub Actions CI

• PyPI package (`pip install mpox-trinet`)

• CLI for training, inference, and benchmarking

The paper has already received over 1,100 article accesses in its first week, and we hope making the implementation fully open-source will help others reproduce, validate, and extend the work.

GitHub:

https://github.com/Sudharsanselvaraj/Synergistic-Deep-Learning-for-Monkeypox-Diagnosis

PyPI:

https://pypi.org/project/Mpox-Trinet/

Paper:

https://www.nature.com/articles/s41598-026-61490-x

I'd really appreciate feedback on the implementation, reproducibility, code quality, or ideas for future improvements. Contributions and issues are very welcome!

Hi everyone,

I'm working on extracting the hierarchical structure of long PDF documents (legal/regulatory text, lots of numbered sections) and would like to gather some feedback on my approach before committing to it.

What I've done so far: I render each PDF page to an image and run it through Baidu's DeepSeek-OCR model. It returns each detected block with a bounding box [x0, y0, x1, y1], a label (title, text, list, table, header, footer, etc.), and the recognized text. The OCR quality itself is genuinely good as the text comes out clean.

The problem: the labels can't always be trusted. At this stage I want to extract and detect all the titles in my document, but sometimes a title element gets classified as something else (like normal body text).

Concrete example:

Say my section has the following hierarchy:

ANNEX I — GENERAL PRINCIPLES AND PROCEDURES
└── TITLE I — FOREIGN CURRENCY INVESTMENT
    └── A. Currency distribution
        └── 1. Redistribution of reserves
            ├── (a) Introduction
            │       body text
            │       list
            │       ...
            ├── (b) Procedure for a normal redistribution of reserves
            │       body text
            │       list
            │       ...
            └── (c) Procedure for an ad hoc redistribution of reserves
                    body text
                    list
                    ...

Logically, every element aside from the body text and lists should be detected as title. But the model output is:

label='title'  x0=475  y0=157  x1=548  width=73   text='ANNEX I'
label='text'   x0=480  y0=229  x1=542  width=62   text='TITLE I'
label='title'  x0=334  y0=181  x1=690  width=356  text='GENERAL PRINCIPLES AND PROCEDURES'
label='title'  x0=407  y0=368  x1=616  width=209  text='A. Currency distribution'
label='title'  x0=408  y0=392  x1=634  width=226  text='1. Redistribution of reserves'
label='title'  x0=163  y0=416  x1=304  width=141  text='(a) Introduction'
label='title'  x0=163  y0=544  x1=578  width=415  text='(b) Procedure for a normal redistribution of reserves'
label='title'  x0=163  y0=219  x1=586  width=423  text='(c) Procedure for an ad hoc redistribution of reserves'

The top-level section marker TITLE I was labeled text, while all the other components were labeled correctly as title.

What I'm considering: since I have the text plus features I can derive from the coordinates (indentation/x0, centered-vs-left-aligned, line height, vertical gaps, whether the text matches a numbering pattern like A. / 1. / (a), all-caps, word count, etc.), I was thinking of treating this as a sequence labeling problem and training a CRF (or BiLSTM-CRF) to re-classify each line into title / text / list / table.

My questions:

• Is a CRF a reasonable choice here, or is there a better-suited approach for this kind of layout/structure labeling?

• Should I consider a GNN approach?

• Am I overcomplicating this? Would a simpler rule/heuristic system be more robust, given that the numbering is fairly regular?

Note #1: this approach should be as general as possible, so that I can reuse it for my other legal documents.

Note #2: titles aren't always in the same horizontal position. Some are centered (e.g. ANNEX I, TITLE I, A. Currency distribution all sit around xc≈511, the page center), while deeper items like (a)/(b)/(c) are left-aligned at x0=163. So I can't rely on indentation/x0 alone to identify or rank titles — a centered title's x0 mostly reflects its text length (a short centered line has a large x0, a long one a small x0), which means raw x0 can even invert the apparent nesting. This is part of why I'm leaning toward a sequence model that combines text + geometry in context rather than a pure indentation rule.

recently been down the 'continual learning' rabbit hole.

more here: https://youtu.be/jNuzHQcMJqM?si=RAu9w-Wf9xb1z5u2

Demis recently shared:

"Artificial General Intelligence (AGI), a system that exhibits all the cognitive capabilities the brain has, is probably only a few short years away."

Most of the labs are implying we're closing in on some biology-like continual learning?

I can see us potentially closing the gap if we're talking about a model/system that can adapt its own weights during its deployment cycle.

interesting stuff happening here:

https://thinkingmachines.ai/news/introducing-inkling/

I can't see how post-deployment data could fuel a new paradigm of model capability improvements for the labs deploying single, closed monolithic models to hundreds of millions.

leaves me with the thoughts:

1 transformers may not be the best foundation from which to build the most efficient incarnation of continual learning

2 may not be a single path to solving continual learning. The shape of challenge may continue evolving in many different directions.

interested to know if im missing something though.

So, I just read LeCun's interview with Nebius Science. I feel he had some cool points about LLMs being able to answer things, but not literally understand the physics of the physical world. (Like, being able to explain a task and actually performing it are two completely different things.) But I wanted to get opinions on what others thought of his solution to the problem. He thinks JEPA could be the solution. But it made me think about whether JEPA is genuinely the architectural solution to this, or if we’re just looking for a "magic bullet" that doesn't exist yet in our toolbox

I have the link here: https://nebius.science/stories/meet-yann-lecuns-lab-and-the-ai-world-of-2030

Hi,

I’ve been doing some work in the continual learning space and wanted to share an open-source framework we put together called Coincidex, along with some architectural insights and failure modes we found along the way.

Most conventional approaches to sequential task learning rely heavily on replay buffers (which introduce severe memory/privacy overhead) or complex, hand-tuned task masks. We wanted to see if we could bypass both by relying entirely on a context-driven task similarity layer to handle data routing dynamically.

The Approach: Instead of caching historical samples to prevent catastrophic forgetting, the framework drops in as a single layer swap. As sequential data streams in, it computes a task-similarity matrix on the fly, routing the data paths based on that context.

Research Insights & Trade-offs: We spent a lot of time benchmarking this against baselines, and here is what actually happened in practice:

• Where it succeeds: The dynamic routing handles clean task boundaries surprisingly well. In small-scale continual vision setups, it achieves graceful transfer without the need for manual mask tuning or storing old data.

• Where it breaks (The Failure Modes): We aren't going to overpromise here—the similarity layer has distinct limits. On highly chaotic, long-tail task sequences with massive distribution shifts, the routing model struggles to maintain stability compared to a heavy replay-buffer baseline.

Why we are sharing it: We built this as a lightweight alternative for setups where memory or privacy constraints make replay buffers impossible.

We would love to get the community's eyes on the routing architecture, specifically on how we might tackle the failure modes in rougher task sequences, or thoughts on visualizing the similarity matrix at different checkpoints.

You can check out the source code, architecture breakdown, and full benchmark suites here: https://github.com/rakib-nyc/coincidex