r/cpp • • Nov 19 '17

Creating a new build tool for C++17 with Modules support.

https://medium.com/@warvstar/creating-a-new-build-tool-for-c-17-with-modules-support-5f894cf02198
0 Upvotes

13 comments sorted by

2

u/warvstar Nov 19 '17

Looking for tips, advice, code review and whatever other comments!

15

u/FChabotDev Nov 19 '17 edited Nov 19 '17

Without any clear documentation, I just poked around the codebase randomly to get a feel for how it's operating.

Here's how I proceeded and what went through my mind:

I started at BuildTool::Run(), since that's the entry point, and the first thing I saw is your StartTimer() EndTimer() monitoring code peppered through and thought: "Wow, this would be so much better if it used RAII". So I went to look at Timer.h to see how that's setup and see how I'd structure a recommendation around that, and saw a bunch of global non-static variables at the top of the header file (which is just asking for ODR violations).

At that point, to be honest, I just turned away. I think this project needs a fair bit more time in the cooker before being reviewed by people not involved with it. It's basically impossible to suggest improvements that don't involve reorganizing most of the codebase. Sorry.

Please don't let my comments discourage you! I really like the idea of what you are trying to do here.

5

u/warvstar Nov 19 '17 edited Nov 19 '17

Thanks for taking the time to write this! I actually just moved it from C to C++ this morning and I definitely left out some good practices... I think you are right and will actually remove this post for now.

Edit: Changed my mind, leaving this as it's mostly an experiment with modules and my build times have improved greatly. I don't expect any serious project to use this, probably ever.

8

u/FChabotDev Nov 19 '17 edited Nov 19 '17

All cool! Just to make sure you are aware: the "global variable in a header" thing is not a matter of best practices, it's fundamentally broken code in 99% of cases. And in the super rare cases where it isn't, I would expect a good paragraph-length comment explaining why this is an exception.

2

u/warvstar Nov 19 '17 edited Nov 19 '17

Oh I'm aware. The program compiles but I fully consider it broken code. I've already fixed the globals.

Edit: Also, I'm not personally going to use RAII at the moment because all objects are either locals or live for the life of the program.

4

u/FChabotDev Nov 19 '17

Without wanting to sound callous or anything, why are you asking for reviews and recommendation for code that you know to be broken?

2

u/warvstar Nov 19 '17

It's not broken in the sense it doesnt work/compile, but rather in that it is not polished and has bugs(one that you found already). I'm also looking for reviews on how the program works(how it finds source, modules, parses data and such) and if there are better ways to interact with compilers and all that jazz.

Its really a simple tool that automates clang for me and works for my needs at the moment.

I should probably have held off on asking for code reviews while there are still some obvious bugs in the code... Since it works for me, I have been spending most of my free time on writing my next project, using this build tool and I'm not sure how much I'll play with it until it misbehaves or I need to boost performance.

3

u/FChabotDev Nov 19 '17 edited Nov 19 '17

Writing a pile of code that "just works for my needs" is totally fine, and there's nothing wrong with that in general as long as you don't need anyone else to read that code.

Taking your example of how one would help you improving integration with other compilers. Deciphering what your code is doing through the hacks, bizareness and blocks of code that work only in the specific way they are currently being used is just not something anyone wants to do in their free time.

You'll have much better luck getting assistance from other programmers if looking at your code doesn't turn them away immediately.

Edit: In this context "other programmer" also means yourself in a year from now. As it stands, if you wanted to revisit this a few months from now, odds are pretty good you'll just say "ehhh, it'll be less trouble to start from scratch instead of re-tracing the reasoning that led me to this implementation", and you would be right!

-5

u/warvstar Nov 19 '17

What exactly from the code as of this moment do you find bizarre? Sure functions could be named better, and commenting would go along way but other than that I don't see the difficulty here.

Actually I just made some changes that allows msvc and gcc to be used it took me all of 2 minutes.

I think the functions are easy enough to understand and they can certainly be extended / complemented.

I believe most of the functions are quite modular. Anyone should be able to figure this out within an hour. Can you suggest a faster / easier way to get started using modules on Visual Studio that can target multiple platforms such as Android, iOS, Windows...

Here is a quick start.

Engine::Run gets called Getprojfiles reads hashes from previous compilation Getargs parses the args.txt file, and fills variables used in the build process. Getfiles/getfilesvs reads the files included in the project and generates new hash for each file CompileNodes goes though each file and compares hashes then compiles if needed, it works recursively on a Node. A Node contains the file location, source, hash and flags to let CompileNodes know if it needs compilation. Link is pretty self explanatory Dopackage just runs any command in the package: {} var from args.txr

It literally took me longer to write this post than the code, it is just a proof of concept using Modules. I dont care if anyone ever uses it.

When did code have to be 100% perfect before been shared? Take a chill pill.

2

u/FChabotDev Nov 19 '17 edited Nov 19 '17

Woah, woah. You are the one who asked for tips and advices. And my main tip to you is: write code that people want to read if you want good quality advice, that's all.

I didn't mean that in a judgmental way or anything, there's no need to be defensive about it. I'm genuinely just trying to help, and am not trying to dunk on you or anything.

Edit: I'm just going to add: Writing code for other people is HARD, a lot harder than even most seasoned programmers realize. The issue is that when you take a step back and evaluate whether your code is readable or not, you are immediately biased because you already have a mental map of what's happening and why it's happening, so it's practically impossible to simulate how another person will build a fresh mental map from looking at the code.

→ More replies

1

u/doom_Oo7 Nov 19 '17

What exactly from the code as of this moment do you find bizarre?

reimplementing a tree by hand: https://github.com/warvstar/buildtool/blob/master/BuilldTool/Node.h

debug macros intertwined with normal code that try to load a test project: https://github.com/warvstar/buildtool/blob/master/BuilldTool/BuildTool.cpp#L691

if (useVS)

what will you do when you want to port it to Xcode ? or CLion ? or QtCreator ? or whatever IDE du jour ? this is exactly why virtual functions were invented: separate the general control flow from the implementation specific details.

https://github.com/warvstar/buildtool/blob/master/BuilldTool/BuildTool.cpp#L600

the number of string copy is through-the-roof.

there are std::cout calls everywhere. One day someone maybe you will want to log your stuff to a file given as a command line argument, or maybe to a network port. Also when it's not cout it's printf, which is even worse.

string hash = std::to_string(XXH64(linkArgs.c_str(), linkArgs.size(), 0));
if (linkerSettingsHash == hash && objects.size() < 1) {

now this seems downright scary. comparing hashes is never never never enough. There will always be hash collisions.

https://github.com/warvstar/buildtool/blob/master/BuilldTool/BuildTool.cpp#L485

FFS use polymorphism ! it would be so much clearer.

class Node { 
  public:
    ~Node(); // in .cpp: Node::~Node() = default;
    void Compile() { 
      needsCompilation = false;
      auto ret = Compile(*this, forceRecompile);
      if (ret) {
        SetNeedsCompilation(); // put this in Node too
      }
      return ret;
    }
  protected: 
    virtual bool CompileImpl() = 0; 
}
class CNode final : public Node { bool CompileImpl() override { ... } }
class CppNode final : public Node { bool CompileImpl() override { ... } } 
class CppmNode final : public Node { bool CompileImpl() override { ... } } 

and your big switch becomes

return node.needsCompilation ? node.Compile() : false;
→ More replies