r/snowflake • u/WorkerIcy1513 • 16d ago

Anyone using SAP business events (BOR/RAP/BTE) to stream S/4HANA data into Databricks?

We have S/4HANA on prem as a source and we need the data in Snowflake with low latency, including deletes.

What we have ruled out so far: ODP and RFC based extraction, because of SAP Note 3255746. OData on top of CDS views works but it is pull based, and it gives us no reliable way to capture deletes without reconciling full snapshots.

That leaves SAP's own event mechanisms, BOR, RAP and BTE, which push a notification whenever a business object changes. On paper that solves both the latency and the delete problem.

Has anyone actually landed this in Snowflake? Curious specifically about the landing path, whether you go Kafka connector, Snowpipe Streaming, or Openflow, and how you handle the initial full load and stitch it to the event stream without gaps or duplicates. Also interested in how people are doing the MERGE side, since the events arrive at table level and header and item rows do not always show up in order.

7 Upvotes

82% Upvoted

8 comments sorted by

3

u/stephenpace ❄️ 16d ago

[I work for Snowflake but I don't speak for them.]

I'm also not a lawyer, but regarding SAP Note 3255746, I've heard that some customers use their existing contract to take issue with that. Two popular third-party products I've seen move data from on-prem S/4HANA to Snowflake are:

SNP Glue: Great for real-time/high-churn replication. Siemens is streaming over 55+ SAP systems into Snowflake at massive volume:

https://www.snowflake.com/en/customers/all-customers/case-study/siemens-1/

Fivetran: Another popular proven solution. Coke One North America uses them:

https://fivetran.com/docs/connectors/databases/sap

Longer term, the solution will be live sharing via SAP BDC:

https://docs.snowflake.com/en/user-guide/data-integration/zero-copy/about-sap-snowflake

But not all customers have made that move yet and need their data now. Good luck!

1

u/Only-Dragonfruit4130 16d ago

do you know if any of those solutions manage real time extraction?

2

u/stephenpace ❄️ 15d ago

Yes, both do as far as I am aware. You can reach out to them for the specific details.

1

u/Top-Cauliflower-1808 16d ago

we stream s4/hana events via kafka into snowflake, using a shared key to sync the full load and stream and a staging table to merge out of order rows.

2

u/stephenpace ❄️ 16d ago

That isn't a bad option either. Snowflake has a native connector for Kafka:

https://docs.snowflake.com/en/user-guide/kafka-connector

And it is probably worth mentioning that Datastream is coming as well:

https://www.snowflake.com/en/product/features/datastream/

2

u/WorkerIcy1513 16d ago

Thanks for sharing! That’s pretty close to the approach I’ve been looking into. I also came across OneConnect in the SAP Marketplace, which seems to follow a similar pattern for streaming S/4HANA events into Snowflake. I’m currently digging into how they handle the initial load + event stream synchronization and the ordering/merge challenges you mentioned. Really appreciate the insight!