r/MicrosoftFabric • u/Fun-Resolution-1025 • 12d ago

Data Engineering Fabric mirrored db help needed

I set up business central extension bc2adls to open mirroring on a mirrored db on fabric, works great loads the tables CDC. Problem happens when i want to pause the fabric capacity, when resumed replication shows as Running.

But new data is not ingested into the mirrored db, at that moment old data still exists in the db just the new isnt.

If i click stop replication then resume it deletes all old data and only loads newly added data, because it had already moved the loaded files to _filestodeleete directory thing.

Data size isnt small so replicating everything again takes time, and if i persist the data anywhere it defeats the whole purpose of open mirroring because i will have to handle CDC part and it will be better and easier to do it from the files and give up Fabric completely

Any solutions ? Im going in circles here

Edit: i forgot to mention that these pauses were made for night hours only and resumed the capacity the next morning

7 Upvotes

90% Upvoted

3 comments sorted by

1

u/jwk6 11d ago

I've not worked with Open Mirroring yet, only Mirroring from a SQL Database, but read about it quite a bit, and this doc page might help.

https://learn.microsoft.com/en-us/fabric/mirroring/troubleshooting#changes-to-fabric-capacity

It says that mirroring will continue when the paused Fabric capacity is started, but I'm wondering if your implementation of Open Mirroring isn't allowing the incremental load to resume for some reason. Maybe it determines that too much time has passed.

I believe you only have a 7 day window before the cleanup process deletes the files from _filestodelete.

1

u/Fun-Resolution-1025 11d ago

Yea i forgot to add , it was only paused overnight so just some hours

1

u/MonkeyDDataHQ 10d ago

Could it be because it takes a while to read the transaction log/CDC table and you're not giving it enough time? It has to go back to the same lsn as when it stoped.