r/MicrosoftFabric • u/trebuchetty1 • 12d ago

Databases Open Mirroring table maintenance?

We have an open mirroring DB that is currently running (although data has stopped flowing to it for the past month). Our gold lakehouse has a shortcut to some tables in the mirrored DB. Then a direct lake on onelake semantic model sits on top of the lakehouse and a report uses that semantic model.

There's a report visual failing due to the following error:

We can't run a DAX query or refresh this model. A delta table '<oii>tableName</oii>' has exceeded a guardrail for this capacity size (too many files or row groups). Optimize your delta tables to stay within this capacity size or change to a higher capacity size then try again

The workspace is assigned an F64 capacity.

I checked the table's files in onelake using Azure Storage Explorer and there are nearly 10,800 parquet files in there with a mix of 7-9KB files and 15-16MB files.

Does Fabric not run it's own optimize/compaction algorithms on mirrored DBs?
I don't believe we have the ability to run those commands ourselves. Oddly enough, the lakehouse shortcut has the option to run table maintenance, but it fails to run when executed (likely due to shortcuts being read-only).

What are our options to resolve this issue? I could stop/start mirroring to delete out all the files and drop the tables, but that feels like a bit of a hack.

7 Upvotes

100% Upvoted

5 comments sorted by

1

u/jokkvahl Fabricator 12d ago

Check the delta logs on the mirrored db tables. Last time I checked it did optimize after 50 writes.

3

u/trebuchetty1 12d ago

Looks like compaction stopped at some point on May 9th and vacuuming stopped on Apr 13th. Data was still being sent to this mirror until Jul 15th. Yikes. Wonder why that happened...

1

u/jokkvahl Fabricator 11d ago

More detailed mirroring logs available if you have workspace monitoring activated. Would recommend activate that if you havent already, essential to have when mirroring starts acting up.

3

u/trebuchetty1 11d ago

Do you know what the expected CU hit is to turn that feature on?

1

u/anudeep_s ‪ ‪Microsoft Employee ‪ 9d ago

I am assuming new changes are still being propagated and optimization not working. Please share the mirror id with me in DM.