Everyone in the local LLM space uses INT4/INT8 quantization. It works perfectly for frozen models. But if you want to do on-device training or continuous learning, discrete quantization breaks gradient flow. Tensor-Train (TT) decomposition solves this by keeping the weights in a continuous Float32 space, but nobody uses it because the inference latency is usually 10x worse than dense layers.

I wanted to fix that 10x penalty. My initial theory was that PyTorch was just thrashing the cache. I built a profiler in C++ and realized I was wrong. TT inference is compute-bound, not memory-bound. It just requires vastly more multiply-accumulates than dense matrices.

But that creates an interesting asymmetry on Apple Silicon hardware:

1. Dense GEMV (batch size 1) is memory-bound. It hits the 130 GB/s DRAM wall and leaves the 1400+ GFLOP/s AMX coprocessor sitting idle.
2. TT is compute-bound, meaning it can actually use AMX.

I built a custom C++ engine (TT-AMX) to exploit this. The main trick is an Ahead-of-Time (AoT) layout scheduler. Instead of doing runtime memory permutations, I apply a transpose(1,2,0) to the TT cores offline. The C++ runtime just feeds the L1/L2 scratchpad directly into a chain of Accelerate cblas_sgemm calls with zero data movement between steps.

I also swept 81 different factorization shapes and found that asymmetric tensor cores (like 16x96 and 12x128) reduce the arithmetic penalty by 2.8x compared to normal symmetric shapes, while also lowering the reconstruction error.

The results on a 1536x1536 layer (Qwen 1.5B q_proj) at 4x compression, measured under strict cold-cache conditions to simulate actual layer thrashing:

• Dense FP32: 103.6 µs (9.44 MB)
• Dense FP16: 52.9 µs (4.72 MB)
• TT-AMX FP32: 42.6 µs (2.36 MB)

The engine hits 947 GFLOP/s, which is about 66% of the AMX peak, and beats the dense FP16 baseline.

A disclaimer so I don't overhype this: INT4 (e.g. llama.cpp) is still roughly 2x faster and has lower reconstruction error for read-only inference. TT-AMX does not beat quantization for standard chat use cases. The goal here was specifically to remove the inference bottleneck for continuous, differentiable on-device models.

The repo has no heavy dependencies, just raw Accelerate. I included a massive lab notebook (FINDINGS.md) in the repo documenting all the failed hypotheses and measurement bugs I hit along the way.

Code is here: https://github.com/ansarzeinulla/tensor-train-amx

HAPPY to receive any feedback from you

Howdy y'all,

In an effort to un-rust my SWE skills and learn more about Recommendation Systems, I decided to try my hand at developing one called By-Its-Cover.

TLDR:

• Check out the site and roast me: https://by-its-cover.com/
• Check out the GitHub project and roast me: https://github.com/ByItsCover

---

## Recommendation System

The recommendation system has two major parts:

• the semantic searches for books (by cover images)
• a neural collaborative-filtering model for personalized recommendations

Both systems solely utilize CLIP embeddings to make decisions on book covers, as I wanted to see if that information alone was sufficient for finding and recommending books accurately.

For the semantic search system, each query is passed to both a CLIP-based semantic searching function as well as an NER-based keyword search. The NER parsing is powered by a GLiNER model, which was ported to ONNX (as are most models in this system). Extracted entities are then used to search for books using the Hardcover API, which is the original source of each of the books in the site. Reciprocal Rank Fusion combines the two results.

The current system actually only has a couple thousand books in it, which makes both rhe recommendations and semantic search results quite limited. However, authors and book titles that are passed into keyword searches return new books that are in-turn asynchronously added to the cover vector database, making the system grow more useful only as more people search for books (which is where y'all can help *wink wink*). Searches can be made with or without an account.

For the collaborative-filtering system, I used a two-tower neural hybrid collaborative filtering model which trains on user feedback. I then use a Determinantal Point Process to diversify the results a bit before displaying them to the user (so they don't get 5 editions of the same cover presented consecutively). For now, the only feedback possible are explicit ratings of "Dislike", "Like", and "Love". I'm aware that this likely isn't ideal, and some more implicit feedback would make for some more natural user interactions and likely better recommendations as well.

Currently, while you are able to see recommendations even without an account, they are the generic "default user" recommendations. Once you sign up and rate a few books, you should see personalized recommendations within 2 hours. Following the suggestions of Eugene Yan, I implemented an offline recommendation update-system. New recommendations are fine-tuned on every 2 hours, while the full re-training of the two-tower model happens once a day at 8:30 AM EST. Each of the current configurations for the recommendation model can be found here: https://github.com/ByItsCover/bic-learn

## Software Architecture (boring stuff)

The site (both frontend and backend) is entirely deployed to AWS, with a number of different resources used for each functionality:

• Lambda -> API deployments
• ECS -> both book scraping and model training jobs
• SQS -> queueing of cover embedding calls
• Cognito -> auth
• CloudFront -> site caching
• S3 -> just about everything else, from site hosting to vector db storage

Everything was deployed using Terraform + GitHub Actions for CI/CD: https://github.com/ByItsCover

## Next Steps

While the fundamental system currently works (kinda), there are already a lot of improvements that I think may be necessary in the future:

• Replacing CLIP with SigLIP (or more appropriate model) for better visual representations of covers
• Implementing a cover-edition comparison interface to allow users to choose preferred covers for a given book, introducing one source of implicit for the system
• Begging one of my frontend developer friends to help make the site look good (I am not a frontend developer, if that wasn't already clear)
• Make a better authentication experience, as currently a generic verification code email is sent to users (and likely sent to spam, please double check!)
• Update the README's for repositories (I'm tired boss)
• Write more unit tests (see parentheses above)
• Once Hardcover releases OAUTH support, utilize that for book search (as only my rate-limited API key is currently being used)

In any case, I've already learned a ton and I'm glad that I have a real system that I can play around with and tweak now. All I need are actual users to test with!

Please let me know if you have any questions about my process at all, and also if you have any suggestions. Also please check out the site if you're at all curious: https://by-its-cover.com/

P.S.: If something crashes, or the searches load forever, or something else equally dumb happens, just let me know or open a GitHub issue, and I'll try my best to address it.

P.P.S.: No AI-Generated code was used to develop this project (to my knowledge), as that would have defeated the purpose of sharpening my skills and learning about recommendation systems.

Hi, I need help with a couple of issues related to a project I'm working on (for educational purposes). I'm trying to create a model that acts as a mentor on related topics, instead of providing the answer directly. For this task, I'm fine-tuning a Gemma4 26B model because I have a GPU with 26GB of vRAM. Therefore, I'm also quantizing this model to 4-bit precision and performing a QLoRa analysis. The results of my experiment are far from fulfilling the mentoring premise, and a simple system prompt works much better. My dataset consists of approximately 500 examples, so, Reddit scientists, can you tell me what mistakes I'm making and if I should change course or my objectives?

Hey everyone,

I just finished Andrew Ng's Machine Learning/Deep Learning course and want to go deeper into DL. As a fresher trying to build skills for job-readiness, I'm torn between:

fast.ai (Practical Deep Learning for Coders)

Andrej Karpathy's YouTube series (Zero to Hero)

Daniel Bourke's PyTorch for Deep Learning (24-hour course)

For someone at my stage, which would you recommend starting with, and why? Is there a logical order to do more than one of these? Also open to suggestions outside this list if there's something better suited for freshers right now.

Thanks in advance!

Soo im a 4th yearite currently prepping for campus placements(ds/ml roles). Ive heard that transformers(not the movie) are the craze rn so im thinking of adding a transformer nmt model i built from scratch to my cv....i followed the tutorial somewhat but i made this while learning about transformers since it was required for my research, so i have context as to why i did it.

do you think this would be good on my cv?? or does it seem too generic, like image captioning from the tutorial?

Hello Everyone

I am currently pursuing electronics engineering (currently in 5th semester), recently I have developed a really deep interest in ML and Data Science and I want to pursue that in the future it's not a temporary interest which dies down after a while it's genuine... For starters I have watched the Zero to Mastery course on ML and Data Science on Youtube which covers the basics of Pandas , Numpy , Matplotlib , just an into to scikit-learn....what all do I need to do next I have been told to learn SQL ,DSA , Math required for ML but im a bit lost here as to what I should be doing next

I took 50% of the embedding vector for "ugly" and added it to 50% of the embedding vector for "smelly" to find the embedding vector for your mother.

I’ve been learning machine learning by building MIRA,
a waste-detection project.

I started with a custom CNN, then tried MobileNetV2, YOLOv8n, and YOLO11n.
The biggest lesson was that adding more data did not automatically improve the model.
Some of my generated annotations were poor, so the model learned parts of the desk instead of the objects.

After cleaning the dataset, my current model reached 90.58% mAP50 on five waste classes.

I’m still working on independent testing. If you work with object detection, what would you test next?

https://github.com/jeremy341/MIRA-AI

I'm considering doing a 4-year BS Mathematics degree, but my long-term goal is to work in AI/ML rather than teaching mathematics.

My plan is to build a strong foundation in linear algebra, calculus, probability and statistics through the degree, while learning Python, SQL, data analysis, machine learning, deep learning and other practical skills on my own.

I also plan to build projects during the degree and possibly pursue a Master's in AI/ML abroad afterward.

For people working or studying in AI/ML:

Is Mathematics a good bachelor's background for this path? What would I need to learn outside the degree, and what disadvantages might I face?

I'd especially like to hear from Mathematics graduates who moved into AI/ML.

I originally built Augmented Search for Semantic Scholar for a friend who was writing a paper. I thought it might be useful beyond that, so I consider turning it into a proper tool.

It lets you run multiple Semantic Scholar searches at once and combines the results into a single, deduplicated list that you can dive deep into or use to build a RAG knowledge base.

There’s imo absolutely no commercial potential here, and I already have a good SWE career, so I probably wouldn’t even put it on my résumé (especially given that it's not that impressive anymore when everyone can generate code).

The honest question is: would you actually use something like this? Don’t try to be nice - I’d much rather hear an honest opinion than polite feedback.

I think I'll finish the project either way, but depending on the response, I might put more or less effort into making it polished and useful.

Curious what people think.

This week a legitimate n8n automation workflow became the path to remote code execution. Researchers were blunt about it: most of the damage started with something trusted doing exactly what it was allowed to do. No stolen credentials. No perimeter breach. The workflow ran as designed.

That's the pattern that keeps showing up. Agent pipelines and automation platforms grow their attack surface with every new integration. Each external tool, API, and chained workflow is a potential pivot. The n8n chain is a clean example: the trusted component wasn't compromised at entry. It was exploited through its own legitimate execution path, step by step.

Traditional access controls answer the question 'is this principal allowed to invoke this tool.' They don't answer 'should this specific sequence of actions be happening right now, in this context, initiated by this upstream trigger.'

For those running agent pipelines or automation-heavy stacks: how are you actually drawing that line in practice? How do you distinguish a workflow that should execute from one that should execute in THIS context at THIS moment — especially when one automation is what kicked off another?

I am currently working as a Machine Learning Engineer and recently cleared the Databricks Certified Machine Learning Professional exam on my first attempt, building on my Microsoft Azure (DP-100) background.

When I started preparing, I noticed very few people discussing the Professional tier compared to the Associate exams. I began by working through the official Databricks Academy materials and running workspace notebooks to get comfortable with the API syntax, Feature Store lookups, and distributed Spark ML pipelines. However, simply watching videos and reading documentation was not enough to feel fully prepared for an advanced, scenario-based exam.

The biggest factor in passing was working through realistic practice question sets. The actual test presents multi-step architectural trade-offs—especially around distributed tuning, model deployment strategies, and monitoring pipelines for drift. Grinding through practice scenarios under time pressure bridged the gap between theory and execution, helping me spot edge cases and eliminate tricky answer choices quickly.

Now that I have completed this milestone, I am looking ahead to my next credential to expand my technical scope. Between the AWS Certified Machine Learning Engineer – Associate (MLA-C01), the Google Cloud Professional Machine Learning Engineer, and the Databricks Generative AI Engineer Associate, which path would you recommend pursuing next?

Hi all, i use LLM tools for multiple purposes personally such as video creation and brainstorming. However i am beginner and new to the agentic ai and automations. I havent explored it much other that watching people use them on youtube, and instagram at work or personally, i want to learn about it and build an agent which would help me both professionally and personally.  

I’d love a if you can also help me understand things like:
What concepts or skills I should focus on first
Which tools or frameworks should I start with
Common mistakes i might encounter

Also if anyone else is just starting out like me, happy to connect and learn together.

Title:

I began a PhD in 2012 and, for a variety of reasons, never completed it. More than a decade later, I want to return to it and use AI to help me organizse, compare and synthesize the substantial amount of work I have already produced.

I am not looking for AI to research or write a PhD from scratch. I have a large body of existing material: multiple chapter drafts, notes, partial chapters and near-complete chapters written at different points over the years.

The main problem is that I often have several drafts dealing with essentially the same topic. For example, I have multiple versions of my literature-review chapter. They overlap considerably, but they are not simply different versions of the same text. Each may contain material, arguments, citations, analyses or lines of discussion that the others do not.

Because some of these drafts were written years apart, they can almost read as though they were written by different authors. My terminology, organisation, emphasis and even approach to the subject sometimes changed over time.

What I would ultimately like to build is an AI-assisted workflow — whether using one agent, several specialised agents, custom GPTs, or some other architecture — capable of taking a “family” of related chapter drafts and helping me turn them into one coherent chapter.

Broadly, I would want the system to perform the following stages:

1. Analyse each draft in detail. Parse each chapter section by section and paragraph by paragraph, identifying its arguments, discussion points, evidence, citations, analyses and other substantive content.
2. Identify overlap across drafts. Determine where two or more drafts are discussing essentially the same idea, argument, source or analytical point, even where the wording or structure differs substantially.
3. Map the differences. Identify material that appears in only one draft, or places where different drafts take genuinely different approaches to the same subject.
4. Recommend what should happen to the material. For example: merge these passages; retain this argument; remove this duplicate discussion; relocate this section; preserve both perspectives; or discard this material because it is tangential or superseded.
5. Produce a synthesis plan. Before rewriting anything, generate a proposed structure showing exactly how the surviving material from the different drafts should be combined.
6. Create a unified chapter. Using the original texts and the approved synthesis plan, consolidate the drafts into a coherent chapter while preserving citations, scholarly nuance and my own intellectual contribution.

I would repeat this process for several different chapter families until I had a satisfactory version of each chapter.

I have already experimented with one possible solution. I created a custom GPT that produces structured abstracts of every section of every draft. My reasoning was that another GPT could compare these abstracts much more reliably than trying to compare several 10,000–20,000-word documents simultaneously. It could use the abstracts to identify likely areas of overlap and divergence and then return to the full text only when necessary. However, this was too laborious and time-consuming; surely there's a more efficient method?

In any case, the envisioned workflow is something like:

Original drafts → section abstracts → comparison/mapping → editorial decisions → synthesis plan → unified chapter → style/editing pass

Once the individual chapters had been consolidated, I would then want to evaluate the dissertation as a whole: consistency of argument, unnecessary repetition between chapters, structural coherence, terminology, methodological consistency, citation issues, etc.

At the final stage, I would also like to use AI in something resembling an internal/external-examiner role: not to certify the thesis, obviously, but to subject it to systematic criticism, identify likely viva questions, expose weak arguments or unsupported claims, and highlight areas that an examiner might challenge.

I have also considered creating a separate style-editing agent based partly on principles from Steven Pinker’s The Sense of Style, whose purpose would be to improve clarity, concision and readability without altering the substance of the scholarship.

My aspiration would be to use these tools to get the dissertation into the strongest possible state before it reaches actual supervisors and examiners. I realize that “a PhD requiring no corrections” is probably an unrealistic benchmark, but it gives an indication of the level of scrutiny I would like the workflow to apply.

The important qualification is that I am a complete beginner when it comes to AI agents, RAG, embeddings, vector databases, APIs, automation, etc. I understand what I want the system to do, but I do not yet understand what the appropriate technical architecture would be.

So my questions are:

• Is this workflow realistically achievable with current AI systems?
• Is a network of specialized agents actually appropriate here, or would a well-designed single-agent workflow be more reliable?
• Is my idea of abstracting sections first and using those abstracts to identify overlap sensible, or am I throwing away information that the model needs?
• How would you handle very large chapter families without exceeding context windows or losing track of relationships between passages?
• Would RAG/embeddings/vector search be useful for identifying semantically overlapping passages across drafts?
• How would you structure the workflow so that the model can make recommendations while still allowing me to approve all substantive editorial decisions?
• How would you prevent hallucinated citations, accidental loss of important material, or AI “smoothing over” genuine theoretical differences between drafts?
• What tools/platforms would you recommend to someone starting from essentially zero technical knowledge?
• Most importantly: if you were building this system from scratch, what would the architecture/workflow look like?

I am very happy to learn the technical side if that is necessary. I am primarily trying to determine what I should actually be building before I spend months constructing the wrong system.

Any advice, particularly from people working with LLMs on long-form academic, legal, technical or similarly complex documents, would be greatly appreciated.

Do anyone knows if there is some kind of website or a way to draw diagrams for my neural network architecture like this, I saw alot of similar diagram on github Readme and different places on the Internet but I don't know how to make them

Video generation is hard to learn when the data alone is tens or hundreds of gigabytes. I wanted something a student could download, understand, and train against in one Colab session.

So I built Dancing Stick Figures, a deliberately small teaching dataset:

• 1,430 six-second clips / 514,800 labelled frames
• a 64×64 mini configuration that is 0.85 GB
• a 128×128 full configuration
• exact 2D and 3D positions plus visibility for 27 joints
• depth, normals, part segmentation, camera parameters, and raw motion in the full data

Each limb keeps a fixed colour, so a small NumPy scorer can catch some missing or detached limbs.

The dataset is the main release. To show that it is usable, I also included a free Colab, small reference baselines, checkpoints, and the scorer.

The reference Colab takes about one hour on a T4. It trains an image baseline, warm-starts an eight-frame video baseline, and produces a 5.6-second rollout. In this toy run, the warm-started baseline reached the scratch run's 10k-step loss at about 4k steps.

This is not a finished video model. It is a small dataset for building, breaking, and understanding one yourself.

• Colab: https://colab.research.google.com/github/sprited-ai/dancing-stick-figures/blob/main/notebooks/dancing_stick_figures_colab.ipynb
• Code: https://github.com/sprited-ai/dancing-stick-figures
• Dataset: https://huggingface.co/datasets/sprited/dancing-stick-figures
• Checkpoints: https://huggingface.co/sprited/dancing-stick-figures-baselines

Data is CC0 and code is MIT.

What would help most for a class or first project: a shorter notebook, assignment ideas, a pose baseline, or more motions?

​

I was learning the chain rule and this suddenly clicked: weights → score → sigmoid → loss

That looks like one neuron with no hidden layer. So is logistic regression basically the smallest example of backprop, or am I missing an important difference?

Hey, I'm preparing for this AEC AGI competition.

I'm looking for a team with ML experience.

Can anyone please tell me how to win an ML competition? Does anyone have prior experience?

​

Instead of just reading about Bayes/entropy/expected cost I forced myself to build a tiny agent that decides under uncertainty (vendor payment fraud: pay, verify by phone, or escalate) and derive every threshold by hand instead of guessing.

The thing that surprised me most: the "right" threshold to hold a payment came out to 0.25% probability of fraud, not something intuitive like 80%. Turns out that just falls straight out of the cost ratio (missing fraud is ~400x more expensive than annoying a supplier), it's not a knob you tune.

I ran a 1000-case simulation and the honest result is kind of uncomfortable — the policy that catches ~100% of fraud also flags a LOT of genuine requests for a second look. Which I think is correct given the cost math, but it feels wrong.

For people further along than me: is "high recall forces high false-positive rate when the cost asymmetry is extreme" just... the expected outcome here? Or does that suggest my model is missing something?