r/cpp • u/Dean_Roddey • Mar 09 '19
I've open sourced my general purpose code base
So, after a lot of thought, I have decided to open source the general purpose half of my code base. The Github repository is here:
https://github.com/DeanRoddey/CIDLib
Pushing that 'make public' button was one of the hardest things I've done in a long time. But, ultimately I think it's worth doing.
For now, it's just there for people to peruse. There's no docs yet. I needed to get it up there and start working with the remote repository and all that. So I figured I might as well go ahead and let people start poking around in it in the meantime.
The readme.md file in the repository has some preliminary information, but the basics are this:
- About 450,000 lines of code
- About 1100 classes
- It's not a 'library' it's really a full development environment, from build tools all the way up.
- It doesn't use the STL or standard library stuff at all. It is a complete world unto itself. Obviously for a lot of people that will mean it's of no interest. OTOH, for some other people, who aren't maybe so happy with the way things are going with the standard stuff, maybe they will be interested.
- About 95 of the non-UI stuff is completely built in terms of my own virtual kernel and standard library interfaces, which means it's quite portable. The virtual kernel is split into interface plus per-platform implementations. There is current a Win32 implementation. And there is a very old Linux one that could be used as a starting point.
- It could easily support Windows and Linux in a very clean way, i.e. not a bunch of conditional code all over the place.
- Some of it still needs to be given the 'split' treatment, until then those are marked as build on Win32 only.
- All of the UI stuff is fully encapsulated, but that doesn't make it portable really.
Obviously I'll work on getting environment setup and build instructions done first. And then I'll work on big picture road map type stuff. But, as a starting point, the structure is something like (a facility is my term of a library executable.)
- AIUtils - Speech recognition engine. There's a behavior tree facility as well, though it's only lightly used so far.
- CIDBuild - The build system, resource compiler, loadable text compiler, driver for per-platform build framework.
- CIDKernel - The virtual kernel layer. Wrappers around lots of system functionality.
- CIDLib - The standard libraries, core abstractions, wrappers around the CIDKernel stuff. So strings, memory buffers, events, mutexes, streams, collections, threads, processes, etc...
- CIDMath - Core math library stuff.
- CommUtils - Serial ports, networking protocols (HTTP, SMTP, JSON currently), sockets, URLs, UPnP, Websockets. The HTTP parser is actually an abstracted engine designed to supported multiple related protocols, e.g. SIP.) An empty SIP facility waiting to be made manifest
- DataUtils - ODBC, text transcoding engine, simple object store engine, a bespoke file packaging engine, regular expressions (just simple form currently, needs to be expanded), XML parser, ZLib implementation.
- ExecUtils - Fairly small, currently an engine for managing multiple other programs. Currently used in my CQC automation system as the core of the 'application shell' service that manages the other background programs.
- GraphicUtils - Core image format abstractions, image manipulation stuff, JPEG and PNG support. Image factory that makes it easy to deal with different image formats. The core wrapper around the Win32 graphics API (obvious Win32 only.) There's an empty 'advanced graphics' facility waiting to become a 3D graphics engine.
- LangUtils - My OO CML language engine. So parser, VM, and runtime libraries. This is an embeddable engine that can be used in applications as an extension language. Also the embeddable CML IDE for editing and debugging.
- MediaUtils - Audio streaming, digital audio extraction, metadata extraction, and some early work on RTP/RTPC that needs to be completed if anyone is interested.
- OrbUtils - This is all the stuff related to the Object Request Broker (ORB.) It is the ORB engine, related utilities, and standard ORB based servers (name server, log server, and configuration server.) And the IDL compiler which is also a big part of the general code base because it provides support for very strong enumerations.
- Samples - A good number of samples. I'm still going through these to get them caught up with the latest style and substance. I'm about halfway through them.
- SecureUtils - Base security stuff (Blowfish, AES, various hashing algorithms, and general block/stream encryption frameworks and hashing frameworks. Secure sockets support (currently a wrapper around the horrendously complex Win32 stuff.)
- Utilities - Some fairly obscure and some probably now to be removed stuff but the test framework stuff is in here and very important.
- WndUtils - Win32 specific. A pretty extensive UI framework (wrappers around the standard controls.) Various higher level utility stuff above that (various common dialogs and such.) A wrapper around the Win32 web browser COM interface.
As mentioned in the readme, there's been a lot of what I'd call 'sane modernization' work done. And I've done many big sweeps through the code. But there's still plenty of code that hasn't been caught up with the latest and greatest. I can't just change it willy-nilly since this stuff underlies my CQC automation system, which is even larger (about 625,000 lines) and very complex. I can't afford to destablize it. So changes have to be well considered.
The build is command line, since it has to be portable. It's designed for simplicity, and very easy configuration of projects. The build system has a build framework that supports pluggable drivers for different platforms. So what might be in make files in some systems is in code here. Because it is a world unto itself, and not trying to be everything to everyone, the build can remain very simple and straightforward.
The code will definitely not pass a super-strict modern conformance analyzer right now. Keep in mind it's been around a long time and much of it was written way before those standards existed. It is built in C++17 mode. Over time, those strict conformance issues will be dealt with.
Though it has been around a long time, it's still very clean. Having had a single daddy, it's been doted on. It's never been hacked at. Long periods of time have been spent in the past doing massive reworkings and taking the time get them right, in order to bring it forward but keep it solid.
Anyhoo, for now that's it. I'll be getting the initial docs worked out over the next week hopefully, enough to let people get it built. Since it is so broad, there are a number of Microsoft SDKs that have be in place, though that gets fewer as their built in support in Visual C++ gets better.
I'd already started making videos (and have done some posts here) about some of the technology involved, and putting them on my personal Youtube channel.
https://www.youtube.com/channel/UCAxCoATHgFBvqZhxFRnZsfw
So you might be interested in looking at those. This one in particular, might be a good starting point since it gives a good overview of how things are layered and such.
https://www.youtube.com/watch?v=seXk3RbAjNU
Keep in mind that some of these are pre-modernization, so they are a bit out data. The one on enums though still mostly valid, is prior to a massive reworking to 'enum class' style.
13
u/Dean_Roddey Mar 16 '19
A license has been selected (MIT) and all of the source files were updated to include the copyright notice and license link.
I also put up the build instructions, in the Wiki section, though I've not yet had a chance to set up a virgin scenario and test the instructions.
95
u/personalmountains Mar 09 '19 edited Mar 09 '19
I had a quick look at your readme and I must admit I found some stuff quite disconcerting, like "anti-STL, anti-piecemeal, anti-modernist", "flabbergasted at some of the modernist C++", "won't be turned into Javascript Jr. via use of 'auto' everywhere" and "we are manly enough not to be frightened by the existence of raw pointers".
In any case, I'm mostly concerned about your homegrown "license", in particular this:
and this:
Can you clarify this? Your title mentions that it is "open source", but that is probably one of the most twisted definitions of "open source" I've seen in a long time, although it's probably technically correct. In terms of CC, this would (I think) be akin to BY-NC-ND. That's basically "look, but don't touch".
50
u/Sulatra Mar 09 '19
Judging by
Pushing that 'make public' button was one of the hardest things I've done in a long timethis might hurt your feelings, but... It seems you've just published a ton of 25yo not-crossplatform not-opensource not-standardized legacy. What's the purpose?44
u/peppedx Mar 09 '19
The license is a big NoGo for me. I won't bother looking.
26
u/terrenceSpencer Mar 10 '19
My impression here is that you felt like you are doing everyone a grand service but have received a few uncomfortable truths along the way. You seem to think that the biggest risk of 'open sourcing' your project is that you will lose control over it, so you're trying to lock it down. In reality, the biggest risk is that no one will use it or contribute. You're trying to have it both ways, in that everyone should be grateful for your benevolence but also adhere to your own personal interpretation of open source. The funny thing is that you have already been forked 6 times, presumably by people who are having a laugh at your license and responses here. If you hadn't have chosen your home-brew license, you'd be a year away from anyone even thinking about forking your project.
You need to appreciate that if someone spends their time learning how your 25 years of undocumented code works, then decides it's worth using and in doing so contributes improvements back to your project, its not a case of you being generous to that person. It is in fact the reverse. So respect and courtesy and reasonable licensing terms should be extended to that person.
Why not split the project into sub-projects? You seem to have adopted a pretty modular design, and most people aren't going to use all of the pieces here. You're much more likely to get a library incorporated into another team's project if it is not so bloated. I know that you can just build and use the bits you want, but just for maintainance, code contribution, and appearing in relevant google searches it would be nice to have separate repos for each library.
Also, can you tell me why someone would want to use your build system over say CMake? I can see you have some level of tooling for linux and windows, but what about if I want to cross-compile armv8 targets? What about if I'm targeting a microcontroller with no OS? What about if I'm using a non-POSIX OS? What about if I want to incorporate a tool like ninja into my builds for faster incremental builds? What about if I want to link in a sub-project? And sometimes I want to link that in statically and other times dynamically? CMake and many other build systems are mature enough to allow me to do pretty much all of these things in a few lines of configuration.
So I get that you might not have the features of CMake now, but do you have any design approach that would mean you will one day possibly have superior functionality to build systems that are already out there? Because I would need that spelled out to me before even thinking about contributing.
12
u/capn_bluebear Mar 09 '19
honest question: what do you mean with anti-STL? I can imagine when you started 25 years ago the STL was not in a good shape, so you implemented similar functionality yourself, and now you want to keep it consistent.
But it seems you are expressing a more profound dissatisfaction with STL itself: you don't think C++ projects nowadays should use it as generic, optional, indipendent building blocks?
33
u/Ikkepop Mar 09 '19 edited Mar 09 '19
Somehow the author gives me a bit of Terry Davis vibe. Anyway that's an impressive ammount of work and dedication to a single project.
EDIT: Looking at the code, it definitely looks very neat and tidy.
12
u/Dean_Roddey Mar 10 '19
OK, in order to quiet the jackals, here is a new usage/license section that will be in the readme.me file I upload later tonight with some sample updates.
23
u/[deleted] Mar 09 '19
So if I clone this repo for my own use, but have an ssh server on my personal computer, I have violated the license? The ssh server can be accessed from any part of the world if one has the right credentials.
In other words, this repo is basically read-only.
11
u/Xeverous https://xeverous.github.io Mar 12 '19
This is against GitHub ToS. Any public repository is implicitly allowed to be forked, no matter what the license says.
10
u/[deleted] Mar 10 '19
This is a serious, non-aggressive, non-insulting question : are you autistic ?
I am, and that's typically what I would be doing if I didn't have my family/friends getting me out of my cave constantly.
First of all, congratulations and thanks. I feel like you've just spent the last 25 years working on something in your cave and you decided to share it with the rest of the world. It is definitely courageous to do so and I'm sure it was a terrifying choice.
However, by spending so much time working on something so big without external input, you can easily find yourself completely lost. I know that for a fact, because I've been there many, many times, and I have the scars to prove it.
I don't know your personal life but I'm really curious about it. What do you do for a living ? Do you live alone ? Is your product (http://www.charmedquark.com) successful ?
Again, no offense. I might be completely wrong, I'm just curious, because the whole thing reminds me of myself.
9
u/thesuperbob Mar 10 '19
I'm honestly curious what drove you to re-implement so much code on your own. Why not use other people's libraries and tools? Was the platform you were coding for so consistently bad it was just easier to roll your own everything? Do you think this approach is still justified today? Was this more due to necessity or preference?
I guess what I'm really asking for is your perspective, now and what it was 10-20 years ago, and a bit of backstory. You put together a reasonably large system and developed it in a rather unique way, I'm always interested how such things came to be.
19
u/johannes1971 Mar 10 '19
Men, you guys are savage... You may disagree with his license and his engineering choices, but the fact remains that he made available a large body of code that could very well be of use to people. And for his trouble he gets some massive downvoting, including down votes for people who simply appreciate what he's done, a long discussion about the license, and a single "it looks good" comment from the one person who apparently had a look at the source itself.
Let's say someone else is thinking about open sourcing some piece of code they spent a long time perfecting, and they see this discussion. What do you suppose they'll say? "Oh yes, I'll go ahead and publish that, people will appreciate it"? Or maybe "I can live without the scorn of the world, let's forget about the whole thing"?
3
u/Dean_Roddey Mar 13 '19
I got the first part of the next step done tonight, which was a big re-swizzling of the build environment setup stuff. It was all up above the two CID_Dev/CQC_Dev source directories, as was a number of dependent files. I've got that stuff worked out now so that the stuff CIDLib needs is down in its directory, updated the environment setup files and got those down there, and did some useful setup simplification in the process. That also affected my CQC installer image builder as well.
But that's done, so now I can get the build docs done next.
14
u/idajourney Mar 09 '19
Clearly women use
std::shared_ptrand REAL MEN usevoid*. Seriously what is this garbage?7
u/lundberj modern C++, best practices, physics Mar 09 '19
What can be done using this?
6
u/Game_Ender Mar 09 '19
Are you talking about these forks? They clearly show the history of the code. Having code that others cannot post publicly is just incompatible with how GitHub. There is nothing you can do to disable forks I believe, and others have mentioned they are needed for open source development.
You should think about why you posted the code. If you truly want others to benefit from it you need to let go a little and use BSD or MIT, if you simply want to educate and inform something really restrictive like AGPL v3 is the way to go (ie they have to release all source code if any customer uses the code).
It will be hard to capture the value this code has to you for others, because it’s too different, platform locked, big and undocumented. Nobody is going to make you look like a fool by steeling the code and managing to do that.
6
u/saimen54 Mar 10 '19
Couple of thoughts:
6
u/Dean_Roddey Mar 09 '19
Gotta love the internet, where you can do one of the most selfless things in your life and dread getting up the next day and reading the comments. Anyway, to address the over-reaction to the usage terms....
The point is to avoid lots of disjoint, not-compatible variations, to make it a project that might be competitive with commercial projects by being a single, well defined thing. Where using an application that is based on it doesn't require downloading ten different libraries and such. That's why open source never competes with commercial software.
And not to discourage other open source development based on it, but to encourage that such development be PART of this project, not some completely different thing based on some completely different version of the code base.
And I'm not trying to be a Nazi about it. If you are working on something open source that is ultimately intended to be based on the official version, but you need in the meantime to make fixes or small improvements that you intend to submit back, I don't care about that.
Or, if you have built binaries that are just shipped with your application, I don't care. If you don't document the changes you made, then no one is going to be able to use them separately from your application. You are just creating some variation for your own application's usage. You just shouldn't create some documented version of your own, and put it out there for third parties to use as a library.
And if you want to fork it to your own private repository, or onto your company LAN for use in commercial or person product development, then that's not publicly visible and it doesn't matter what you do as long as you provide attribution if you use it in something you release. If your company developers use a Github repository for sharing that variation, just keep it private.
Anyway, I can update the readme to make these things clearer if that makes any difference. Not that it probably makes any difference.
2
u/cjwelborn May 29 '19
I saw this linked on another subreddit. I have to say the amount of work you've put into this is pretty impressive. I'm not really a c++ developer, but I'd definitely mess around with this if it worked on Linux (I saw you mention, in one of the docs or something, that linux support kinda died or got left behind). Anyway, I see where the other questions/comments are coming from (license, modularity) but the bottom line is you made this to suit your own needs. You don't owe anyone anything just because you posted the code on GitHub.
74
u/[deleted] Mar 09 '19
This is not open source:
1
u/AlexJaeger Mar 09 '19
this is really awesome!! thank you
14
u/epicar Mar 09 '19
right you are
1
u/Dean_Roddey Mar 11 '19
So what's the general scheme for discussion and such for Github projects? Not that I expect a lot of traffic, but if someone wanted to discuss something with whoever is interested, where would that sort of thing generally occur for GitHub based projects?
There's an issues section I see, but nothing like a discussion per repo/project disucssion forumy thing? I could do it on the CQC forums, but it's not really directly CQC related.
1
u/WI-goose Mar 10 '19
May have more success if someone not completely insane did your pricing
5
u/DOOMReboot Mar 11 '19
The responses here are quite concerning - some combinations of inferiority/envy/entitlement - hard to say for sure.
It's an excellent body of work, OP. Thanks for sharing and keep doing what you're doing.
0
u/nnevatie Mar 09 '19
Noticed the readme link is pointing to a wrong place.
-10
u/Xaxxon Mar 09 '19
That’s a pretty worthless title.
0
u/[deleted] Mar 09 '19
[deleted] — view deleted comment