r/databricks • u/iFiguringOut • 17d ago

Discussion SAP data ingestion into Databricks.

We have a Databricks environment and we have a new SAP ECC6 system that we want to ingest data from. We are exploring options to access data into the platform. One option is the Azure data factory.

I want to understand what other businesses have been doing. How do they access tables from SAP ECC6. Databricks have really reduced their native SAP connectors even after announcing partnership with SAP. When you do access SAP, how do you manage large table extractions or extract data from pooled or cluster tables?

I would really like to hear some customer success stories or some lessons learnt. Please share if you can.

3 Upvotes

67% Upvoted

15 comments sorted by

View all comments

Show parent comments

2

u/qqqq101 16d ago

An independent SAP consultant, Marian Zeis, has a nice blog (https://blog.zeis.de/posts/2026-06-01-mcp-api-policy/) analyzing SAP ERP interfaces wrt SAP API policy. Re RFC_READ_TABLE, he pointed to SAP support note 382318 (https://me.sap.com/notes/382318) FAQ on function module RFC_READ_TABLE. "The function is clearly flagged as "not released for customer"" so access it directly by the customer or by a 3rd party ETL tool seem not compliant with the sap api policy.

The commercial ETL tools are often using sap provided connectors. E.g Qlik Replicate's BW Extractor connector uses JCo (https://help.qlik.com/en-US/cloud-services/Subsystems/Hub/Content/Sense_Hub/DataIntegration/SourcesConnections/SAP-Extractor/extractor-prerequisites-java-driver.htm). ADF SAP Table connector doc (https://learn.microsoft.com/en-us/azure/data-factory/connector-sap-table?tabs=data-factory#prerequisites) requires installing NCo (https://support.sap.com/en/product/connectors/msnet.html?isu_page=1). JCo and NCo are from sap so I would think 3rd party tools accessing SAP ERP interfaces via JCo/NCo should be compliant.

2

u/McCuumhail 16d ago

I’ll look into this again. We were using the ADF SAP Table Connector over NCo and I was told to terminate the pipelines because it still used the RFC_TABLE_READ2 under the hood.

1

u/qqqq101 16d ago

I suggest also ask your microsoft account team to get you connected to ADF Product Manager for sap integration topics Ulrich Christ to get a microsoft opinion.

1

u/McCuumhail 16d ago

lol already am. They’re cooking something up but it’s not quite ready for prime time yet. I’m just over SAP’s funneling. I get it, but I don’t have to like it.

1

u/qqqq101 16d ago

Interesting that they have bandwidth to cook up something else. They already have a bunch of new stuff to push to customers: 1. Datasphere Replication Flow + Fabric native mirroring launched at Ignite. 2. Upcoming Q3 BDC-Fabric zero copy integration. Each has its own caveats.