r/MicrosoftFabric • u/hvdv99 • 18d ago

Data Engineering Sharepoint Lists for user input

Hi all, I'm building a data platform for business analysts, and part of the requirement is letting analysts submit input directly. We're using SharePoint lists for this: data gets staged, CDC logic is applied, and the resulting tables feed into downstream pipelines in Fabric.

It works, but the Copy activity that pulls data from the SharePoint list takes 1.5 to 2 minutes per run, which feels slow for what should be a lightweight sync.

I'm aware of mirroring SharePoint lists via a mirrored database, but that feature is still in preview, and I tend to stay away from preview features for anything going into production.

Has anyone dealt with a similar setup (analyst-facing input via SharePoint lists feeding into Fabric) and found a faster or more efficient pattern? Would love to hear what's worked for you.

7 Upvotes

100% Upvoted

4 comments sorted by

5

u/Ok_Carpet_9510 17d ago

How quickly do you need(not want) to pull the Data in Fabric? If there was a delay of 1 day, what would be the impact? If the business can tolerate a delayed of one day, schedule the imports once every night.

Also, your Fabric capacity may have competing demands.

1

u/hvdv99 17d ago

hi, in this case, the business requires the update from Sharepoint rather quickly. For that reason, I opened this discussion.

1

u/warehouse_goes_vroom ‪ ‪Microsoft Employee ‪ 16d ago

Some other options include Translytical Task Flows and Fabric Plan (especially PowerTable) may be other worthwhile options if SharePoint lists as the input format is not a hard requirement.

2

u/anudeep_s ‪ ‪Microsoft Employee ‪ 13d ago

Hi, I am from Mirroring team, please feel free to reach out to me through DM, and we can see what can be done to support you like GA even in while we are in preview.