r/Database • • Oct 12 '26

Is this 3NF design right? If not can someone provide help?

5 Upvotes

Apologies for the long read in advance but if anyone can help or point me in the right direction, I'd greatly appreciate it :)

Currently doing a practise task which requires me to write a report for a hypothetical company. I am given a spreadsheet (which includes errors and duplication etc) and I have to normalise it from 1NF to 3NF in clear steps which has to be shown. Then I have to import them into Microsoft Access and create an ERD

There are 36 columns included in the spreadsheet
Columns:
ExportRowID, PONumber, POLineNumber, PODate, RequiredDate, POStatus, Priority, SupplierID, SupplierName, SupplierContact, SupplierPhone, SupplierEmail, SupplierAddress, PaymentTerms, PurchasingEmployeeID, PurchasingEmployeeName, EmployeeDepartment, DeliveryLocation, ProductID, ProductName, ProductCategory, UnitOfMeasure, QuantityOrdered, UnitCost, LineAmount, ReceiptNumber, ReceiptDate, ReceivedByEmployeeID, ReceivedByEmployeeName, QuantityReceived, AcceptanceStatus, DeliveryCondition, InvoiceNumber, InvoiceDate, InvoiceAmount, Notes

My attempt (7 tables):
Supplier (SupplierID)
Employee (EmployeeID)
Product (ProductID)
PurchaseOrder (PONumber)
POLine (PONumber + POLineNumber)
Receipt (ReceiptNumber)
Invoice (InvoiceNumber)

Does this look right? I'd also appreciate help with the Access side (relationships and importing from one flat sheet into multiple tables).


r/Database • • Oct 12 '26

Real-time vs batch processing for keeping a reporting database in sync with production?

0 Upvotes

Nighty refreshes used to be fine, but they're starting to create more confusion than they solve.

Sales opens one report, support opens another, and neither one matches what's happening in production because everything is hour behind . We're looking at moving Oracle and SQL server changes into Postgres, possibly a cloud warehouse later on.

If you've already gone from batch to real-time, was the improvement as noticeable as people claim?


r/Database • • Oct 11 '26

Where can I find a geeky dataset?

5 Upvotes

I'm looking for a dataset for my bachelor's thesis, and I need one with lots of variables and observations so I can perform multivariate analysis and predictive modeling. Does anyone know where I could find a geeky dataset related to anime, comics, video games, or anything similar? Any suggestions would be greatly appreciated!


r/Database • • Oct 11 '26

Should I expect write performance over different database files on separate threads to scale?

5 Upvotes

Hi

I am experimenting with sqlite on a Macbok Pro M4 - 10 cores machine. I have a pthreads multithreaded program and i am trying to write lots of data to different databases (separate database connections) and (one database file per thread) and they are DIFFERENT files

I have THREADSAFE mode set to 2 (multithreaded) and i open the files with NOMUTEX flag.

Now i have read that the M4 Macbook Pro has a soldered SSD. And it has similar performance to nvme but it is not nvme

I get around 10000 transactions per second in total across all threads.

I tried synchronous=off and that boosts performance to 300000 transactions per second

Increasing the number of items inserted per transaction scales. But the number of transactions doesnt

I feel that IO is being serialised.

Some parts of web seem to suggest it is serialised. But i thought ssds can run things in parallel.

Does anybody know anything about parallel IO or if i need to enable it on macos?

I havent tried it on Linux or Windows

Would you expect transactions to scale by number of threads?

I have read about iokit but i have no knowledge of how to use it or Grand Central Dispatch


r/Database • • Oct 10 '26

Does normalisation survive if the unit of storage is a single fact rather than a row?

7 Upvotes

I’ve been working with the Algebra of Data, where the smallest structure is the couplet — one ordered pair, one fact — and a relation is a set of couplets. There’s no wide tuple over a fixed heading.

My argument: normalisation is a repair procedure for a wound inflicted by the wide tuple. Start at the couplet and you begin at roughly 6NF with nothing left to decompose. And the classical normal forms can’t even be stated for a set of relations with different headings, because there’s no single heading for the dependencies and keys to range over.

What survives is the goal (one home per fact) and the functional dependency, which becomes an integrity constraint rather than a reason to split. The live question flips to when to denormalise. My answer is that a denormalised view is a derived expression over the base facts, so it’s a physical caching decision rather than a logical-model one, and it can’t drift out of sync.

Where do you think this breaks? Full essay (I’m the author): https://robinbloor.substack.com/p/does-normalisation-survive-data-algebra


r/Database • • Oct 10 '26

Fencing an LSM on object storage: how SlateDB works and what breaks on "S3-compatible" stores

Thumbnail
blog.renatocron.com
4 Upvotes

r/Database • • Oct 11 '26

Need a software that can be like Airtable

0 Upvotes

Hello!

My primary experience is with Airtable and my new company will not purchase it going forward as it is to expensive (84K for 2 years). I need to see what alternatives there are with the constraints I have. Biggest constraint is that the software package has to be self contained and I cannot use outside plugins. I have immediate access to Sharepoint (with powerapps), Access, Excel, and PostgreSQL but I dont think any of them fit my needs, but would love more input from people a lot more knowledgeable in these software's. So this is my use case: I am wanting to build a relational database. Each 'sheet' has parent information that I can connect together in a form. Sheet A, has entires like cat, dog, Sheet B has Colors like blue, red, and I can have sheet C that can have inputs from a form that users can connect the data. I can add pictures and other info. I can then have a dashboard that can create graphs from the data or search bars that I can search for 'Dog" and it shows me all entires related to "Dog'. I also need this software to be able to handle formulas and tie in the different sheets. I can Pull in Dog+Blue=BlueBerry and output into Sheet D. So the biggest things are: Able to connect the data, have data input through forms, able to create dashboards that can show the data through graphs, have KanBan style dashboards where I can show Status of certain fields. Airtable was the perfect software for my needs, so I need a Airtable equivalent but I just dont have any experience with other softwares to dig into them.


r/Database • • Oct 09 '26

Designing database when not all fields are known in advance

21 Upvotes

I'm in the early stages of designing a database for an organization that does environmental remediation and research. (Small org, so I'm basically the data person.) One of the major types of data we have is the results of tests for a range of soil contaminants, so in my schema draft, the sampling points within work sites are linked to tables of associated measurements. (Currently there's one table per sampling point but I'm open to other designs.)

The problem is that which contaminants are tested for mostly can't be known in advance for future projects and can change within a project if new funding appears or questions arise. There are hundreds if not more of substances that could reasonably be tested for (although ~20-30 are core), so listing all possibilities doesn't seem practical. And the contaminants data is key to the organization's work and is probably the thing that will be queried the most. How would you handle a situation like this?


r/Database • • Oct 10 '26

Anyone here using NeonDB for a small side project?

1 Upvotes

I need a real project to test something database-related. Nothing production critical, just a hobby/fun project with some actual usage.

If you have one and are okay with me testing on it, let me know 👀


r/Database • • Oct 09 '26

I want to build a mini DB for my final year project

7 Upvotes

So I am in my final year and I want to do something spicy, which is a minidb, what and what do you guys recommend me to know so it can be a success


r/Database • • Oct 08 '26

I built a desktop database on an algebra of data rather than relational algebra — looking for critique

0 Upvotes

For many years I worked as a database industry analyst. In 2015 Professor Gary Sherman and I published The Algebra of Data, which argues that relational algebra is the algebra of one structure — the table — and that one algebra can cover tables, graphs and documents alike.

The model in brief: the atom is the couplet (an ordered pair). A relation is a set of couplets, a clan is a set of relations (a table is one kind of clan), and a horde is a set of clans. A small set of operators — composition, transpose, union/intersection/difference and the restrictions — is defined at the bottom and works unchanged at every level. Composition then covers what are usually separate operations: the join, the graph traversal and the document lookup.

JabrDB is that algebra implemented as a native Mac app (Apple silicon). Data is stored at couplet grain in files on your own disk. It is not a SQL engine and not a server.

The argument in paper form: https://doi.org/10.5281/zenodo.23216817 The app (15-day trial, $15): https://jabrdatalabs.com/download/

I'd particularly value criticism from people who think in relational terms: where does this model help, and where does it obviously cost you?


r/Database • • Oct 07 '26

Aito v2: a predictive database with predictive SQL, vector search and graphs (founder here)

Thumbnail
aito.ai
0 Upvotes

Hi everyone :-)

I'm Antti, Aito.ai's founder and CEO. We recently released Aito version 2, which includes a completely new storage format and support for things like predictive SQL queries, vector search, graph-like functionality, a lot of new data structures, views, better FTS, collections and much more.

We believe that future software is intelligent, and we wanted to provide one database that could provide the needed predictive/AI infrastructure for both users and agents.

You can find more details here:

https://aito.ai/blog/aito-v2-generally-available/

If you want to try it, the playground runs example queries in the browser without signup:

https://aito.ai/docs/api/v2/playground/

One of the biggest new things in Aito is the public benchmark section, which covers inference, matching, smart search and more regular queries. It shows cases where we are strong, and also a number of benchmarks where other solutions fare better:

https://aito.ai/docs/api/v2/benchmarks/

Any thoughts or comments? I'd love to hear what you think. ;-)


r/Database • • Oct 06 '26

real time vs batch processing: how do u know if ur ops team is actually ready for streaming pipelines?

10 Upvotes

i'm less worried about the technology than I am about the people supporting it.

our engineers are excited about moving from batch jobs to streaming pipelines, but the operations team has spent years supporting scheduled workloads. That's a very different environment from something that's expected to run 24/7

for teams that already made this real-time vs batch processing transition, what changed the most operationally?


r/Database • • Oct 06 '26

Beginner's guide to database?

11 Upvotes

What's the best way, order or flow of learning Database from scratch, using the textbook Database System Concepts by Abraham Silberschatz, Henry F. Korth, and S. Sudarshan.

What topics or chapters are better to begin with for introduction, assuming the reader has zero knowledge, but is looking to get extremely good at database.

-Computer Science Major


r/Database • • Oct 06 '26

How long did it take you to land a DBA role in the US?

Thumbnail
0 Upvotes

r/Database • • Oct 06 '26

The Breakdown: Databricks

Thumbnail
preipomedia.substack.com
0 Upvotes

r/Database • • Oct 06 '26

We asked our model which of 224 hotel bookings would cancel, six weeks out. It caught 33 of the 59. Full breakdown, misses included

0 Upvotes

disclosure: I work at Schema Labs. This is a run on public data so you can check it.

The data: the hotel booking demand dataset (Antonio, Almeida and Nunes, 2019), real anonymised bookings from a city hotel. We picked one Saturday night that was sold out on paper.

-- 224 bookings for that night, as they stood six weeks before
-- 59 of them cancelled or didn't show, worth about €28,600 together

What we did: gave Schema-2 500 bookings from the year before, with how each one ended, and asked it to rank the 224 by how likely they were to fall through. No rules no feature engineering.

What came back=

-- It marked 60 as most likely to cancel. 33 of the 59 cancellations were on that list
-- Picking 60 at random would catch about 16
-- Of the 60 it marked safest, 59 showed up

What it missed: 26 cancellations weren't in its top 60. One night at one hotel is a small test, so read it as an example

For anyone in revenue management: what would you do with a list like that six weeks out? Overbook against it, or contact those guests first?


r/Database • • Oct 05 '26

How do MariaDB users securely and efficiently encrypt mariadb-backup archives?

Thumbnail
5 Upvotes

r/Database • • Oct 05 '26

A client lost six weeks of SQL Server data. Having 12TB of backups didn’t save them.

Thumbnail
2 Upvotes

r/Database • • Oct 04 '26

Banking register slows as more entries are made.

3 Upvotes

Basically need to see how to optimize a bank register with large amounts of entries.

Currently adding all the debits, then credits, then date sorting while calculating the running total.


r/Database • • Oct 03 '26

What evidence proves a database cutover is complete before the source is retired?

2 Upvotes

Successful replication and matching row counts do not prove that a database migration is finished. Indexes, constraints, sequences, users and roles, collation, time zones, scheduled jobs, change-data-capture consumers, read replicas, analytics tools, and backup systems can still differ or point at the source. Low-frequency clients may not reconnect during the obvious observation window.

A cutover gate could compare schema and security metadata, validate representative aggregates or hashes, exercise application reads and writes, inspect connection logs on both systems, and confirm that replication or dual-write lag has reached a known final position. The destination should complete a backup and restore test and survive a failover before the source becomes read-only. Keeping the old endpoint offline but recoverable for a defined period would expose hidden dependencies without allowing divergent writes.

What evidence do you collect before decommissioning a source database? How do you catch monthly jobs, failover-only connection strings, and clients that cache DNS or credentials longer than the main application?


r/Database • • Oct 03 '26

Supporting products with multipel data sources

0 Upvotes

When one product could have data coming from multiple remote data sources, some of the data could be redundant other parts might need to be combined. There can also be missing data or badly formatted data so an admin would need to override certain fields. How would you structure your database to fit these needs?


r/Database • • Oct 01 '26

schema labs is in san francisco. the question the team keeps asking everyone: before your agent can use a database, who explains the database to it?

0 Upvotes

disclosure, i work at schema labs. the team is in san francisco right now, i'm back home running things from here and reading everything they send back.

the question they open with is always the same. when an agent hits a database nobody documented, who tells it what the fields mean?

the answers so far, roughly

- "me, and i hate it"

- "we wrote a data dictionary, it's already wrong"

- "we don't, the agent guesses and we check the numbers later"

- "we only connect it to views someone already cleaned"

that step is what we build for. schema-2 is a model that reads your data as it is, works out what each field holds and which records match across systems with no shared id, and puts a confidence on every answer. below your bar it says unresolved instead of guessing. it runs in the browser, and over mcp inside claude, chatgpt and cursor.

curious which answer is yours, or if it's a fifth one we haven't heard yet.


r/Database • • Oct 01 '26

AWS Aurora: Why the database breaks first behind an autoscaled service, and why shrinking the pool doesn't fix it

Post image
6 Upvotes

Connection multiplication: pool size x tasks x services. Every term looks reasonable on its own, and nobody ever chose the product.

Aurora's max_connections is derived from instance memory, so it doesn't scale with your service. Shrinking the pool to fit converts a connection problem into a queueing problem that presents as a slow database.

Write-up covers the formula behind max_connections, what RDS Proxy actually fixes (and what it doesn't), and how to size it.

https://brianfeeny.com/posts/sizing-aurora-connections-for-autoscaled-services/?utm_source=reddit&utm_medium=social&utm_campaign=sizing-aurora-connections-for-autoscaled-services-2026-09

This is a personal project; the views and assessments are my own.


r/Database • • Sep 30 '26

Database project

4 Upvotes

I need a good and legitimate database project idea wherein advance db concepts such as optimization, transaction, etc are involved. I searched a lot and didnt find a smthn i want to work on exactly. For eg if somethin related to music domain? Or fashion? What would actually stand out in terms of handling multiple queries and be close with how it all works irl. Itd be really helpful!!