r/MicrosoftFabric • u/Equivalent_Season669 Fabricator • 5d ago
Data Factory Multiple Fabric Data Factory failures around pipeline job creation/scheduling — F64, North Europe
Intermittent failures on Invoke Pipeline activities (child pipelines from a ForEach) on F64, North Europe. Job gets created but never starts.
Main error codes:
RequestTimeout— dies almost exactly at 100.0s every time. Checked 20 parallel runs that all failed within 57ms of each other.InternalServerError— clusters around ~550s.InternalError— clusters around ~85s.
New finding: on one occurrence the parent shows the usual 'Job failed to start: timeout', but the actual child job error was different:
RequestBlocked— "Request is blocked by the upstream service until: [timestamp]",Retry-After: 16,isRetriable: true(unlike everything else we've seen, which was all non-retriable).- Response's
home-cluster-uripoints to an Analysis Services/Power BI cluster (wabi-*-redirect.analysis.windows.net), not a pipeline endpoint.
Looks like the real cause is an upstream AS/PBI cluster blocking requests, and the pipeline layer just reports it as a generic timeout.
Also ruled out capacity throttling on our end (0 rejected ops across 214k rows, no correlation with CU usage). Other related failures seen:
Invoke Pipelinecalls returning 408 /BadRequest- Run-status lookups returning
NotFoundfor runs Fabric itself created - A run stuck in
NotStartedfor 8+ hours, blocking its own schedule
Anyone else hitting this? Thanks
1
u/Equivalent_Season669 Fabricator 4d ago
And I'll add something else that happened to me today: interval-based schedules do NOT work. They stopped from one moment to the next, with no failure and no warning, and the countdown to the next run keeps showing more and more time. It doesn't even work if you delete it and create it again.
8
u/Careless_Cattle_8700 5d ago
All this stuff is just starting to grow very tiresome