I had posted this in my Build Log, but it was suggested I should also post it here too.
For a long time, I have struggled with how to best manage board revisions and the Gerber files. In the hobbyist domain, it is easy for these to get out of sync, especially if there is some rework on the boards. I had heard of people using git to tag their boards. I also tried this but lacked the discipline to keep this up to date for subsequent changes. After the work I did on the KiCad PCB to Fritzing Part project, I was introduced to GitHub Actions. Some of you might already be familiar with these, but it was this feature that got me thinking about the Gerber management problem. The thought was, could GitHub be used to manage the Gerber generation? So I created this project, kicad-release-automation-module, where a script can be used to install a GitHub action into my KiCad project, and from there, after updating a configuration file, I only need to create a release tag in Git, and GitHub will take care of the rest. The advantage of this approach is that I can use KiCad Text variables to ensure that the release tag is available to silkscreen text. So now the revision that is passed to the fab and the revision that is printed on the board is written in stone and I always have a reference back to that exact revision if I need it.
Thanks Daniel. I have heard of kicad-cli being used to run the DRCs as part of a build process, but I have not seen this process documented - Thinking about it, I guess that would be a further consideration to this action - gate the Gerbers only after a successful DRC.
I am curious as to how I will get on with this approach in the long term. I will also be curious to know how you get on as well.
I’ve experimented with this in the past, but I’m no longer using server-side output generation for the following reasons (most if not all can be worked around, but I found the hassle not worth it):
I find it easier to group multi-PCB projects in the same repository and it just required too much upkeep of CI configuration
I tend to keep project related, non-Kicad stuff in the same repository (LTSpice simulations, calculation tools, bringup/testing notes)- having a fresh set of gerbers for every commit was just confusing (which of them did I really send off for manufacturing?)
At some points, my build process was convoluted (using kicad-nightly for 3d renders but kicad stable for gerbers, as the stable version didn’t have the features yet).
What I eventually settled on is a build.py that I copy over from project to project (as some projects did require slight tweaks), which reads “version” text file next to project files and runs roughly the following sequence:
Creates outputs/project-name-version folder for release
Performs ERC and DRC, failing on any errors/warnings
Zips up the files related to PCB manufacturing (gerbers/drill files) to a single archive
Zips up the files related to assembly (everything except the schematics, step file) to another archive
Resulting outputs directory is then manually reviewed and committed to repository (and tagged)
The software engineer in me does find this approach icky (manual steps, etc)- but the hardware engineer is really glad that there is a single, authoritative, reviewed source for manufacturing outputs for a certain board revision.
Maybe I’m missing something here but, a while ago, I took a look at using github to manage my PCB versions and, honestly, it just seems like a lot of work for little benefit.
What I do instead is keep a separate directory tree for each PCB version. After I send it off for fab, I do a save-as at the project level. This produces a new version in a separate tree that I apply any changes to. If I need need to make an update, I send it off and do another save-as. I always apply changes to the unbuilt version. The storage space isn’t an issue to me but if I need the space, I just zip the directory up. The advantage of this approach is I can keep multiple versions concurrently available. Since I often have several versions in the field that I need to support, that is a requirement.
The one aspect that doesn’t work well for is variants. I have a number of features that are dependant on part placement (via DNP 0R resistors or ICs). But given that is simply a case of manually marking/clearing the DNP check boxes, it is straightforward.
You both raise very good points, especially the concept of having multiple PCBs within the same repository. I have such a repository and this, for me, has issues even at the git branch level. This simple approach seems ok at a basic level but I see your experiences have brought up other cases. This is why I’m interested to see how I get on with this long-term.
Reinis mentions an issue that I came to realise later too - regarding the additional artefacts that get packaged into the release zip. My repositories can be very disorderly.
There’s another dimension I’ve been thinking about that my approach doesn’t really address, which is KiCad’s own evolution. I’ve been caught out a few times trying to open an old KiCad 4 project in KiCad 10 and the libraries don’t always upgrade correctly and there is a bit of rework to get things in sync. These days I keep a few KiCad versions installed side by side and use the project archive feature to zip the project up and tag it with the KiCad version at the time to remind me what version this was created/last touched with. I guess this information could be included in the release notes.
Since I work in the hobby domain, I will continue to test out the GitHub-actions approach but possibly leave out the corner cases or just reorganise my repositories - this might even help me be more disciplined and not litter my repos.
If anyone has other workflows for managing their repositories, I’d be interested in hearing about them.
@Steve_Mayze thanks for sharing this. I haven’t looked deeply into your workflow yet, although I intend to. I haven’t got to a good level of automation yet, although having to work with a larger team recently has driven me to coalesce around a project structure that’s been working reasonably well and definitely needs some refinement. I also use kikit for panelising a design.
I wanted to ask if you explored kicad jobsets while you were building this. It seems to have significant overlap with kibot and is relevant to this topic. I have not explored them yet but if you discounted jobsets I would be interested to hear where they don’t meet your needs.
Great post. Hardware has always been a bit behind build automation and artifacts. It’s probably related to the fact that we don’t generate gerbers as often as the software folks build binaries.
Thanks @smerrett79. It’s good that you mention kicad jobsets. I have to admit I only took a cursory look at them. Since I was only interested in the Gerber generation, I let myself be distracted by the GitHub action and only calling kicad-cli. The kicad jobsets can also be invoked by kicad-cli, so there is no loss there, but they could make my config json redundant. I’ll take a closer look at them again soon.
This thread has actually made me aware of kibot - it looks like there’s a fair bit of overlap between the two, but kibot seems to be the more comprehensive solution. What I’ve put together really only covers Gerber generation.
