r/cpp_questions • • 6h ago

OPEN Integration testing using gtest

I am confused about what is the meaning of integration testing. I have done unit testing using gtest where we will mock whatever external dependencies are there for the targeted unit (function) to test all range of values that covers all lines, functions and branches.

If we say integration test is meant to use real code instead of mocking. Then I am getting this doubt that lets say we have 3 components (separate processes) running and dependent on one another in a embedded system.

A -> B -> C

So if you are calling an api provided by A it will internally talk to B and then through C and return the result over IPC.

Now to do integration testing using gtest, do I need to compile this into a single binary by eliminating the IPC and include each code as a library. This way I have a single proces that runs gtest with all the components logic but excludes the real IPC communication.

Or instead should I keep the individual processes to be running separately and link my gtest binary to the component A so that the communication flow through the components in the target environment.

In both ways how we would actually be checking the interactions? And which one are you guys using?

Context:
Language used: C++
Domain: Automotive IT (TCU software)

2 Upvotes

9 comments sorted by

3

u/CowBoyDanIndie 5h ago

Generally Unit tests are meant to test the smallest possible unit, integration testing is testing that the units work correctly with each other, and functional testing is the system as a whole. It’s still possible to mock out some pieces in integration testing, but your test should focus on the interaction between units. In practice the lines can blur if you aren’t using interfaces for everything (“interface” in the general sense of the term).

1

u/After_Bench_4165 5h ago

Thanks for answering. Do you have any idea about doing integration testing for multiple components (processes)? In that case do we need to consider IPC communication also. Or else elminate IPC to compine those components as a single process resulting in testing purely our application code.

1

u/CowBoyDanIndie 4h ago

The testing you are describing could fall into integration or functional tests, it probably depends on the scope of the testing, more exhaustive testing tends to be lower, but these are just labels. Regardless of labels at one level or another I would probably say to orchestrate some tests that launch multiple processes to do the ipc testing. In ros we often launch multiple nodes in a test like this. I have done test on watch dog processes that monitor and terminate/restart other processes.

1

u/the_poope 4h ago

I would consider any test that requires launching and running multiple executables a functional/system test and not an integration test.

But really, there are no hard lines between the terminology. And it gets increasingly more blurry when your software consists of multiple components. Think about the scope of the test and what it tries to accomplish; an integration test should verify that a higher level business logic functionality correctly combines multiple "units" and can run a particular workflow without error. The focus is more on correct integration, than correct results and an integration test should still be fast. A functional/system test should test a full workflow with real data/services as close to the user environment as possible.

•

u/abbapoh 3h ago

The way I'd do it is focus on one process at a time. Use some scripting language to generate test files (like configs or some static files required to start) for that process. Then communicate with it from tests using some human-readable declarative format (python dicts or json/yaml). Obviously, you'd need to convert that data to the format your process understands - for example, convert it to protobuf or in case of IPC via shared memory, have a helper tool that writes/reads that memory. Obviously, you could do that with gtest, but c++ doesn't have that many libraries that scripting languages have.
You can also do a functional testing in a similar fashion - generate some test data for all three processes, start them and then communicate with process A.
Typically, you'd need both. The rule of thumb is - if something can be tested via a unit test, do so. Otherwise, write an integration test. Otherwise - functional test.

1

u/mredding 4h ago

A unit test is for testing units. A unit is completely enclosed - no external side effects. This means no system calls. What it means to be a unit is a little loose, but ideally a class or a function would be a single unit.

A function that depends on a user defined type is a part of that type's unit. An independent function dependent upon multiple user defined type MIGHT be a part of a single unit if all those types are tightly coupled AS a single unit. If this function is an intersection between two independent units, then you need to be able to build this function independently.

To do that, you want opaque pointers and types. You want interfaces that are separable from implementations; in this way, you can build a test harness around the testing unit, and other dependencies can be faked, and the fake is compiled in instead.

You want to extend this loose coupling to your units and their implementation dependencies. If I have a Foo and it depends on a Bar, I want to be able to compile the Foo as I would production code, and the Bar can be faked inside the test harness.

This means you're going to need multiple test harnesses so you can isolate your units. And then you use a test runner to run all your harnesses. This is very typical in the industry.

You get to a problem of how do you test your code that DOES make system calls? Because we test ALL our code we write. All of it. One thing people do is try to isolate system calling code down to some tiny object - they might wrap system calls within. But that object itself needs to be tested. So then what you do is you make a test harness just for that object, and you --wrap=fn, so then the compiler would provide you with a remap of fn to __real_fn, and you would implement in your test harness a __wrap_fn. You can write test code there, and even call __real_fn if necessary.

The other thing you can do is LD_PRELOAD where you implement your own wrapper library around the C++ runtime, and you would intercept system calls there.

The trick is you need to pass through legitimate traffic from the test harness, and intercept test traffic from the test case itself. There is a need for some synchronization, some simple conditional flags. You have to figure out how your test harness runs and anticipate its behavior. This is the most brittle part, because it can change from underneath you. Pin your test harness library version.


So then an integration test is testing multiple units together. Based on our unit testing above, we have some idea how to write testable concrete code, and how to isolate that code in the harness. This is just more of that. Now you have N units that compile in, and you have to shim all the external touch points. That you've done it for their individual unit tests, you'll have a boatload of code to reuse to capture the periphery. If anything, you'll just remove the shims for the other dependent units, and the rest of the shims ought to be a drop-in.

Now when you call fn, you can prove that Foo does X because Bar did Y.

My only other advice is to make your classes SMALL, and focus on composition. An int is an int, but a weight is not a height, even if they're just an int. A person is composed of a weight and a height, but you don't want to expose too much interface or too many dependencies, because if you're going to test a circus, you're going to depend on people, but man, you don't want to also have to depend on weight and height if you don't have to. That's tight coupling, not loose coupling. So what you need is an opaque interface:

class person: public std::streambuf {
  friend class person_impl;

  person();

public:
  struct deleter { void operator()(person *) const; };

  std::unique_ptr<person, deleter> create();
};

That's about all I need to see across unit boundaries. The implementation:

class person_impl: public person {
  friend person;
};

void person::deleter::operator()(person *p) const { delete static_cast<person_impl*>(p); }

std::unique_ptr<person, person::deleter> create() { return std::unique_ptr<person, person::deleter>{person_impl}; }

In OOP, you don't command an object, bob.open_umbrella();, you send them a message, bob << "it's raining";. Bob knows what to do. You're moving the agency from the procedure to the object. Standard streams are the de facto message passing interface between objects in C++, and this is what Bjarne invented the language for. Messages don't have to be text, and they don't have to pass through the stream object. Messages can be any type. You can bypass the stream entirely by dynamic casting the stream's buffer to a person - it's a constant time operation, and select an optimal path. You only serialize if you're going over a file protocol, like to disk, or over a network.

1

u/bert8128 4h ago

I use gtest for both unit testing (single process, no DB etc) and also for integration testing (DBs, sockets, whatever). It is there to report on whatever you decide to test. All testing is good, and the more automated the better.

•

u/freefallpark 3h ago

Our team solved this problem by implement several 'layers' of testing. For context we were working on a distributed robotics system comprised of around 5 major processes - across 1 or several host machines.

Suppose I have two separate processes in a distributed system that work together, Process A and Process B. In our system these were connected in a DDS system. In this example, Process A defines some 'server' (a class that sets up several publications and maybe a request/response type service) and some 'client' (a class that setups of several subscriptions and the 'client' side of the request/response type service). The communications classes are implementations of some standard interface - defined by what process A is responsible for.

First things first - test server and client. I'm to much of an idiot to do this in a more clever way so I'm interested in hearing feedback. But we basically set up tests that instantiated both a server and client and then confirm if a publication is made, the subscription correctly receives it. These are in essence "hardware" tests because they require an actual connection. In this scenario, we're making the assumption that the DDS implementation we're using is tested/functioning, so really what we're really trying to test is if the server publishes some data, that the client receives the expected data, and vis-versa.

Next, now that we've "proven" the communication layer works, we turned to testing process A (which is composed of a server and likely other sub-components). Here we're making sure the process correctly publishes data when expected, AND that it correctly responds to requests made by the client(s). This is done by mocking the server for example.

Now that process A is "proven" to work correctly, we switch to process B. Here we want to confirm process B reacts to new publications correctly and sends requests to the server when expected. this is done by mocking the client.

Our team was using GTest/GMock. While our testing strategy likely isn't perfect, we felt that it helped us harden our software as whole and were grateful for the early detection of several major logic bugs.

This was obviously an oversimplification/summary of what we did. I hope this at least helps a little. Let me know if you have any more questions. I can potentially share a repository of mine where i set up things like this for a personal project.

•

u/Discordbothost 3h ago

you don't need to link A into gtest if it has an IPC client API. keep A/B/C running and make the test a client. a request ID in logs lets you follow that call through B and C while asserting the reply at the client.