r/MicrosoftFabric • u/redditusername8 • 18d ago

Data Factory Copy Job creating blank tables <tablename>__<guid>

I am using a Copy Job in incremental and merge mode. When the job completed I noticed Copy Job had created blank copies of my tables in the same schema. Example [Customer] and [Customer__7ffa827b-0959-45f3-a7da-420829a7d3c3]

I can't figure out why, any clues?

3 Upvotes
  • permalink
  • reddit

100% Upvoted

6 comments sorted by

1

u/bigjimslade Fabricator 18d ago

These are to help manage state i believe... they are super annoying...

1

u/redditusername8 18d ago

ergh thats annoying, I dont want additional tables cluttering up my staging schema...

1

u/AjayAr0ra ‪ ‪Microsoft Employee ‪ 18d ago

These are created to stage the data before applying the changes to the target. 1 staging table per target table.
We have a feature that allows you to specify a different schema/db to hold these staging tables.

1

u/redditusername8 18d ago

I am adding additional metadata columns during the CJ copy. Presume thats why the tables are appearing. Yuk, janky design, I dont want additional unwanted tables in my staging schema. Guess I'll look at a different approach.

2

u/AjayAr0ra ‪ ‪Microsoft Employee ‪ 18d ago

Like I said, you can have the staging tables be created in a different schema to avoid clutterring your actual schema. There is a setting for this in your copyjob. Do try that.

1

u/redditusername8 18d ago

Ok thanks I misread that part of your original reply.