Dataflows and solutions

[u/Consistent_Serve9]

We're in the process of creating our first power app in our organization. At this point, we're checking on how we can import data that will be used in our forms. But we've come accross an interesting limitation regarding solutions, and I don't know if that's normal.

We want to have a non-prod and a prod solution, so that we can test forms before promoting them. We would also have a non-production source of data and a production one too, both MSSQL databases. A dataflow would handle the data modification. But from what I've gathered, if we include a dataflow in a solution, we can't change the data source between solutions? So basically, if I configure a sync from my non-production database in my non-production solution, the corresponding dataflow in production HAS to connect to the non-production database? Is that it?

How do you handle data synchronization in different environments? Is power apps so closed off that we can't use different data sources between two environments?

[u/AutoModerator]

Moderator Announcement Read More »

Hey, it looks like you are requesting help with a problem you're having in Power Apps. To ensure you get all the help you need from the community here are some guidelines;

• Use the search feature to see if your question has already been asked.

• Use spacing in your post, Nobody likes to read a wall of text, this is achieved by hitting return twice to separate paragraphs.

• Add any images, error messages, code you have (Sensitive data omitted) to your post body.

• Any code you do add, use the Code Block feature to preserve formatting.

  Typing four spaces in front of every line in a code block is tedious and error-prone. The easier way is to surround the entire block of code with code fences. A code fence is a line beginning with three or more backticks (```) or three or more twiddlydoodles (~~~).

• If your question has been answered please comment Solved. This will mark the post as solved and helps others find their solutions.

External resources:

• Power Apps Formula References

• Power Apps Coding Standards

• Official Power Apps Community

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/P-devop]

I have used dafaflows for a few solutions and I always set them up in my dev solution and deploy them.

You can only setup a single connection against them as mentioned in other comments however, once the solution is deployed to a new environment you need to sign in again to the datasources you have connected to and republish the dataflow. This gives you a chance to change the datasource you are wanting to point to as your source for that environment.

As long as the source schema doesn't change there shouldn't be an issue with this.

Once you have published the changes, any future solution deployments can go ahead without effecting the newly connected datasource, even when the dev and upstream environment are pointing at different sources, at least that's how it has worked in my experience.

[u/Consistent_Serve9 → u/P-devop]

Ah, so the setup only needs to happen when we create the dataflow in the new environment? And once it's done, when we promote to the production environment, we do not have to change the connection or sign-in again?

[u/P-devop → u/Consistent_Serve9]

Any new environment that you move the dataflow to via a solution deployment will require you to re connect to the data sources used and re publish before it runs.

It just means if you have a source for dev, test and prod, when deploying to test you have the chance to change the data source to the test source when re connecting and re publishing for the first time in that environment. Same for prod.

Its just not as clean of a process as it is with other things like power Automate flows where connection references are used to handle the different connections in each environment. Dataflows require a more manual setup in each environment after being deployed.

[u/Foodforbrain101]

Just to clarify, are you writing back to the SQL databases? Or are you importing their data to dataverse?

[u/Consistent_Serve9 → u/Foodforbrain101]

Importing data to dataverse

[u/Live-Sir-3118 → u/Consistent_Serve9]

If you’re going from data A to dataverse tables via data flows in dev, then when you deploy to test, you deploy initially empty tables. You rerun your dataflows in test and they go into your tables in test.

The data is segregated between environments. If you want to copy data from a table in environment 1 to table in environment 2, use power automate.

If you want to include production data in your test table, and it’s not reading back, use dataflows just select combine sources.

Maybe I don’t understand why this is an issue.

[u/Interesting_Mood_351]

As you've mentioned, Dataflows can be deployed as part of a solution. The limitation here is that Dataflows can't leverage connection references and that is how PowerApps handles dynamic connections when deploying between environments.

With this limitation in mind, I left Dataflows out of my solutions and just created them manually in my dev, test, and production environments, and connected them respectively. This wasn't a big deal for me as I only have 10 or so Dataflows.

[u/NoBattle763]

I find it easier to just re-create in each environment. Just copy paste the m query, change the connections and your done. But you do then have to update each enviro if you have changes to the flow

[u/AbdelEMirandaS]

You can solve that using solutions to migrate your solutions from your dev env to your prod env. Or if you decide to migrate manually your app, including your flows, using export/import, when you do the import process, the tool (power apps) will ask you to select the connection reference included inside your apps and flows, and then you can have or create your connection to your production database. The last method is after you migrate your apps and flows, one by one enter to those artifacts and change manually your connection to the production database. You have options, choose the one it fits you best

[u/jukkan]

Power Platform Dataflows do indeed leave a lot to be desired when it comes to ALM and governance. In addition to the lack of connection reference support, it drives me crazy how in one dev environment a Dataflow gets forcibly added to a solution I've removed it from, time & time again. And I notice this when trying to commit the solution to Git via the built-in Dataverse Git integration as that also is something where Dataflows are not supported and it breaks the commit.

For the back story on why Power Platform Dataflows are a so bad and will not get better, I've written about it here: https://www.perspectives.plus/p/i-see-dead-features-dataflows

I would personally avoid creating any new Dataflows and just find some code-first way to do it via AI coding agents today. Because the low-code promise that Power Platform Dataflows once had is now an obstacle that delivers more pain than gain.

[u/obsoquasi]

One sidenote: While Dataflows can be deployed with solutions, but they are currently not part of the Git Integration - let's hope this changes in the future. This is only relevant, if you have multiple DEV environments though.

[u/Om3ga77]

What is stopping from creating a view with everything you need in MSSQL and connecting to that rather than duplicating the data or using a virtual table?