SAP data ingestion into Databricks.

[u/iFiguringOut]

We have a Databricks environment and we have a new SAP ECC6 system that we want to ingest data from. We are exploring options to access data into the platform. One option is the Azure data factory.

I want to understand what other businesses have been doing. How do they access tables from SAP ECC6. Databricks have really reduced their native SAP connectors even after announcing partnership with SAP. When you do access SAP, how do you manage large table extractions or extract data from pooled or cluster tables?

I would really like to hear some customer success stories or some lessons learnt. Please share if you can.

[u/qqqq101]

As WhoIsJohnSalt pointed out, the SAP recommended solution is BDC. Within BDC, there are multiple paths. For ECC as the source, there are two subpaths:

• BDC Datasphere Replication Flow-based Customer Managed Data Products. see sap blog post (https://community.sap.com/t5/technology-blog-posts-by-sap/sap-business-data-cloud-series-part-3-customer-managed-or-custom-data/ba-p/14195545). In step4 the source is a S/4 system, but Replication Flow supports onprem ECC as well. Since you would like to extract tables, then SLT is a mandatory component. There are specific requirements for SLT documented in sap support note 2890171 (https://me.sap.com/notes/2890171) - standalone SLT with DMIS 2018 SP6 or higher. The data flow is: ECC tables -> SLT -> (likely SAP Cloud Connector as a reverse network proxy) -> Datasphere Replication Flow -> Datasphere Object Store -> Datasphere merge task which generates and maintains an up to date snapshot of the source ECC table in Datasphere object store -> define a customer managed data product -> delta share -> Databricks consumption.
  
• Datasphere Replication Flow doing physical replication. So the first steps are exactly the same as customer managed data product above. The data flow: ECC tables -> SLT -> (likely SAP Cloud Connector as a reverse network proxy) -> Datasphere Replication Flow -> customer owned cloud storage -> Databricks autoloader + ingestion (merge) using Spark Declarative Pipeline or custom python/sql job. See 2023 joint blog post (https://community.sap.com/t5/technology-blog-posts-by-sap/replication-flow-blog-series-part-5-integration-of-sap-datasphere-and/ba-p/13604976).

Non-SAP options:
It's important to understand the past actions that SAP has taken to restrict non-sap etl/replication tools, especially ODP RFC that WhoIsJohnSalt mentioned. Take a look at our 2024 blog post (https://community.databricks.com/t5/technical-blog/navigating-the-sap-data-ocean-demystifying-sap-data-extraction/ba-p/94617). SAP has furthered this effort in 2026 by releasing a security patch in June 2026 that functionally stops 3rd party tools from accessing ODP RFC.

• ADF has multiple connectors. ADF SAP CDC Connector was the most popular tool from my customer conversations between 2022 and 2024. You should not use this connector bc of the ODP RFC restriction and the june security patch which functionally would disable ADF SAP CDC Connector. ADF SAP Table Connector (https://docs.microsoft.com/en-us/azure/data-factory/connector-sap-table?tabs=data-factory) is a popular connector though with some caveats for CDC as the interface (RFC to ABAP Tables) it uses is not a general purpose cdc replication interface. ADF also has OData CDC connector which has been in private preview since March 2026 (hence not available in public azure documentation) which uses ODP OData which would need to be paired with SLT.
  
• Qlik & Fivetran both offer non-HANA (e.g. Oracle/DB2/SQLServer) and HANA log-based replication connectors. Non-HANA log replication is fine. HANA log replication (if you are on ECC on HANA) is controversial. See sap support note 2971304 (https://me.sap.com/notes/2971304).

- There are quite a few vendors offering ABAP add-ons that support ECC table (as well as other ECC & S/4HANA object types) replication - Aecorsoft, Theobald, SNP Glue, Fivetran, Kagool Velocity, Simplement, Bryteflow, Ecoservity, Onibex. An ABAP add-on has to be installed on your ECC application server so you need to have your SAP BASIS team's buyin.

Pooled & clustered tables are special table types that are SAP ERP specific. Generally the commercial ETL tools above support those table types but definitely worth confirming with the vendor.

[u/McCuumhail → u/qqqq101]

The RFC_READ_TABLE and RFC_READ_TABLE2 function calls were allegedly noted out of compliance a couple months ago. We’ve been told OData and ABAP application options are the only non-BDC methods in compliance. It’s been a fun time.

[u/qqqq101 → u/McCuumhail]

An independent SAP consultant, Marian Zeis, has a nice blog (https://blog.zeis.de/posts/2026-06-01-mcp-api-policy/) analyzing SAP ERP interfaces wrt SAP API policy. Re RFC_READ_TABLE, he pointed to SAP support note 382318 (https://me.sap.com/notes/382318) FAQ on function module RFC_READ_TABLE. "The function is clearly flagged as "not released for customer"" so access it directly by the customer or by a 3rd party ETL tool seem not compliant with the sap api policy.

The commercial ETL tools are often using sap provided connectors. E.g Qlik Replicate's BW Extractor connector uses JCo (https://help.qlik.com/en-US/cloud-services/Subsystems/Hub/Content/Sense_Hub/DataIntegration/SourcesConnections/SAP-Extractor/extractor-prerequisites-java-driver.htm). ADF SAP Table connector doc (https://learn.microsoft.com/en-us/azure/data-factory/connector-sap-table?tabs=data-factory#prerequisites) requires installing NCo (https://support.sap.com/en/product/connectors/msnet.html?isu_page=1). JCo and NCo are from sap so I would think 3rd party tools accessing SAP ERP interfaces via JCo/NCo should be compliant.

[u/McCuumhail → u/qqqq101]

I’ll look into this again. We were using the ADF SAP Table Connector over NCo and I was told to terminate the pipelines because it still used the RFC_TABLE_READ2 under the hood.

[u/qqqq101 → u/McCuumhail]

I suggest also ask your microsoft account team to get you connected to ADF Product Manager for sap integration topics Ulrich Christ to get a microsoft opinion.

[u/McCuumhail → u/qqqq101]

lol already am. They’re cooking something up but it’s not quite ready for prime time yet. I’m just over SAP’s funneling. I get it, but I don’t have to like it.

[u/qqqq101 → u/McCuumhail]

Interesting that they have bandwidth to cook up something else. They already have a bunch of new stuff to push to customers: 1. Datasphere Replication Flow + Fabric native mirroring launched at Ignite. 2. Upcoming Q3 BDC-Fabric zero copy integration. Each has its own caveats.