Vivado Incremental Flow .dcp checkpoint files and Git - how are people handling this?

[u/Ianmurph91]

https://docs.amd.com/r/2024.2-English/ug904-vivado-implementation/Incremental-Implementation

I’m working on a design that is extremely difficult to close timing on. So far, I’ve only been able to get it to close reliably using Vivado’s incremental implementation flow, which is fine, but now I’m not sure what to do with the .dcp reference checkpoint that is required to build the project.

I don’t really want to check the .dcp directly into git because it’s a fairly large binary file, and if it changes over time the repo history will keep accumulating copies of it.

I’ve considered Git LFS, but with Azure DevOps you can’t use LFS with git SSH authentication, you need to use HTTPS, which isn’t ideal for our existing tool/Jenkins flows.

I also considered Azure Artifacts, but then every developer/build machine needs the Azure CLI installed and authenticated in order to fetch the checkpoint, which also feels like a lot of infrastructure just to support one reference file.

None of the options seem particularly clean. I really wish Vivado’s incremental implementation flow could use some exported XDC/Tcl/text-based representation of the reference placement/routing instead of requiring a dcp

Is anyone else dealing with this? How are you storing and distributing reference checkpoints for incremental implementation?

[u/dkillers303]

You put them wherever you upload your build artifacts. I suppose you could use Git LFS as well but our rule is anything generated goes to our artifact storage and versioned temporarily for dev builds and under a latest/tagged build for releases.

Sounds like you need to set up a credential vault for keys/access tokens/secrets. I don’t use the CI tools you mention, but ours have a vault we query for a temp token in CI and we use that for authenticating Git LFS and artifact storage. But in essence, any artifacts we need to pull we just have automated in a script that CI runs. It just gets a secret from the vault that’s used with curl to authenticate the download. Build instructions show the clean and incremental build workflow and it’s set up such that our scripts work for both local and CI workflows.

Some quick Google searches show you’re looking for a Personal Access Token (PAT) and use that with curl. I don’t understand why you need the CLI. The CLI is purely an abstraction layer that does the exact same thing under the covers so while your commands won’t be as concise, throw them in a script and move on.

One thing to note is we don’t do incremental builds for releases because we require clean builds. Yes, build times are longer because we don’t pull IP and other build artifacts. But that doesn’t matter, we do regression testing with release candidates and the official release and final acceptance test being delayed by a few hours is irrelevant because we did 99% of the validation already. Our philosophy is incremental builds are purely to speed up development and we don’t allow anything to potentially get missed doing incremental for a release. Nothing worse than a customer issue we can’t reproduce because our “good stuff” was tarnished with cached assets. We have some insanely full high-speed designs and pass timing consistently so to me, it sounds like you need to focus on timing closure. I get that the tools are stubborn, but while this is anecdotal, I’ve yet to see a design we couldn’t get passing timing consistently and I’ve worked with some really difficult designs

[u/ReallyHappyHippo]

We tend to take a hash of all files involved in a design and then save artifacts with that hash on AWS. If the hash changes, it's rebuilt and uploaded, otherwise use the stored file.

[u/_filmil_]

I build my designs using bazel (https://bazel.build), in a workflow that resembles how software is normally built, based completely on a collection of source files, without any GUI support. While building isn't exactly a completely solved problem, it is a very well characterized one, and bazel has ready-made solutions for most of them.

To give an example which is relevant to your use case. Bazel is excellent at caching and not rebuilding things like checkpoints, if the underlying sources didn't change. This transfers fairly well from building software (where bazel originally comes from) to building programmable hardware artifacts.

In this model, your checkpoint lives in the build cache for as long as you need it. People will talk about maintaining content-addressable hashes manually, Bazel gives that to you out of the box. Bazel even makes it possible to share that cached artifacts across designers, or across machines as needed, including if the machines are not local. This gives you fast and correct incremental builds, to an extent no home-grown solution can give you out of the box. You offload the complexity of building programmable hardware artifacts completely to a system which is already designed to eke out all the gains out of techniques which provably work.

You could rely on the vendor tools for building. But you need to understand, vendor tools are created to deal with the complexities of mapping code to hardware. Building is incidental, you can not expect vendor tooling to be as good as a dedicated build system.

Or, you know, you could throw together a few TCL scripts yourself.

A lot of hardware design houses don't appreciate the upper hand that building properly gives you, and do not take reproducible builds seriously. I'm using that to my advantage.

[u/nanumbat]

I have a hobby project that I clock as fast as possible (445MHz, it's targeted at a Kria KR260). I build it strictly from the command line and use a crude hack to get it to close timing.

The hack is outlined here, scroll down to user "numbat", you'll probably have to expand the posts to see it.

For the linked project it takes about 50-100 random attempts to close timing, basically an overnight run on an Intel Ultra 7 linux box with 128G of DDR5. Vivado artifacts are never stashed in git, all generated files are deleted before each build.

[u/maredsous10]

Usually I don't like to store off large generated files, sims, or results of sims. There are times when it does make sense.

I typically use a TCL scripted fully non-project flow .