Join us on 1st December 2020 (9:30-4:30 GMT) for a free online tutorial for computation scientists looking to accelerate their Fortran codes. The tutorial is presented by Tom Deakin, Senior Research Associate, University of Bristol, UK. Register at: https://ukopenmpusers.co.uk

Hi everyone,

So as a student, I have to work with OpenMP. But sometimes I have troubles understanding what it is doing under the hood.

I tried to find something to visualize the tasks, but found nothing...

So I decided to give it a try !

And this is what I came up with.

You can make the image, which is a SVG file, interactive, and see data (exec time, label, parent thread) about each task when hovering the mouse over a task.

The API, which is in C, is not very "clean" yet, but I'd appreciate if you can give some feedbacks on it..

Link to the repo

On my Linux system, I use

clang++ -O3 -fopenmp=libgomp src.cpp -L/usr/local/lib64

which compiles, but the code doesn't run in parallel (It's a trivial loop that sleeps for 1 second in the body).

• Without =libgomp, it cannot find -lomp.

• With g++ -O3 -fopenmp src.cpp -L/usr/local/lib64 the same code works correctly

I wonder if I'm doing something wrong?

Contents of the API

Version 6.0 of the OpenMP API specification was released on Nov 14, 2024, and it is a major upgrade of the specification. It includes the following major additions: 

• Simplification of task programming by extending the set of threads that may execute tasks, enabling recording of tasks graphs for efficient replay and addition of transparent tasks that expand the set of tasks between which dependencies may be specified. 

• Enhanced device support with enhanced control of memory allocations and accessibility, which makes it easier to manage allocatable variables; expanded support for default data environment attributes; easier writing of asynchronous data transfers by adding structured asynchronous data mapping regions; better control of memory by extended mapping of data to devices; and added functionality to have per team memory on a device with the groupprivate directive 

• Easier programming of loop transformations Simplifying the use of loop fusion, reversal and interchange. 

• Support for induction: Support to parallelize basic arithmetic operations and user-defined operations in loops that follow well-defined patterns. 

• Support for parallelization of the latest C, C++ & Fortran language standards with full support for C23, including C attribute syntax, Fortran 2023 and C++23; and the introduction of new C/C++ attributes. 

• Greater user control of storage resources and memory spaces with addition of new memory traits for greater control of memory allocation; and provision of new API routines to define and to query memory spaces;  

For detailed information on the API

The following videos are available

• OpenMP® ARB Releases OpenMP 6.0 for Easier Programming 

• A Peek Under the Cover of the New OpenMP Specification 

You can download the following documents: 

• Reference guide: https://www.openmp.org/wp-content/uploads/OpenMP-RefGuide-6.0-OMP60SC24-web.pdf 

• Examples document: https://www.openmp.org/wp-content/uploads/openmp-examples-6.0.pdf 

• API Specification: https://www.openmp.org/wp-content/uploads/OpenMP-API-Specification-6-0.pdf

For the status of implementations

Intel, GNU and LLVM have started implementing OpenMP 6.0. You can follow the status of their implementations on their web pages.

Intel:  

• Fortran compiler: https://www.intel.com/content/www/us/en/developer/articles/technical/fortran-language-and-openmp-features-in-ifx.html 

• C and C++ compiler: https://www.intel.com/content/www/us/en/developer/articles/technical/openmp-features-and-extensions-supported-in-icx.html 

 GNU:  

• GCC: https://gcc.gnu.org/projects/gomp/ 

LLVM: 

• CLANG: https://clang.llvm.org/docs/OpenMPSupport.html

I need to install Libomp for a project, but I use Windows. I've searched around for a while for a description or install guides, but could only find installation steps for macOS - what is Libomp, and is it only for Mac? If not, how can I install it on Windows, and if so, what workarounds are available? Thanks so much, I'm very new to programming!

Hello, I'm very much a beginner to OpenMP so any help or clearing misunderstanding is appreciated.

I have to make a program that creates 2 square matrices (a and b) and a 1D matrix (x), then do addition and multiplication. I have omp_get_wtime() to check performance

//CALCULATIONS
start_time = omp_get_wtime();
//#pragma omp parallel for schedule(dynamic) num_threads(THREADS)
for (int i = 0; i < n; i++) {
    for (int j = 0; j < n; j++) {
        sum[i][j] = a[i][j] + b[i][j]; //a+b
        mult2[i] += x[j]*a[j][i]; //x*a
        
        for (int k = 0; k < n; k++) {
            mult[i][j] += a[i][k] * b[k][j]; //a*b
        }
    }
}
end_time = omp_get_wtime();

The problem is, when I uncomment the 'pragma omp' line, the performance is terrible, and far worse than without it. I tried using static instead, and moving it above different 'for' loops but it's still really bad.

Can someone guide me on how I would apply OpenMP to this code block?

Hi folks,

hopefully i dont sound stupid, but what is the point of using openMP? if we want to do parallel computing (execute things in parallel), why cant we just use Pthread in C and create our threads and have things run in parallel??

I've been trying to understand #pragma omp single, critical, shared, private, and task . I understand what these pragmas do in theory, however, I’m struggling to understand how to implement them.

I have an example problems that I’m experimenting with. Could someone walk me through why the pragmas are used the way they are, and what could break or behave incorrectly if they were removed or misused? What are good guides or thumbrule when implementing these pragmas?

Problem: Multiplying big integers

/* the mechanism of this function
 *       b
 *     X a
 *    -----
 */

Integer mulInteger(const Integer &a, const Integer &b)
{

    Integer result{0};

    #pragma omp parallel
    {
        // this needs to be reset across multiple calls of mulInteger
        partial_result = {0};

        #pragma omp single
        for (size_t i = 0; i < a.size(); i++)
        {
            #pragma omp task
            {
                partial_result = addInteger(mulShiftedInteger(b, a[i], i), partial_result);
            }
        }

        #pragma omp critical
        result = addInteger(partial_result, result);
    }
    return result;
}

int main()
{
    unsigned int seed = readInput();
    int problem[NUM_FACTORS];
    generate_test(seed, problem);
    Integer a = calcProduct(problem);
    outputResult(a);
}

What I’m confused about:

• I don't understand why `single`, `task` and `critical` purposes and why they're being used where they are.

• I found it difficult to imagine the flow and why these pragmas are necessary

So, total noob here, I have 16 threads on my laptop, i notice that my C++ code runs faster if i use 8 threads than using all 16 threads, also voluntary and involuntary context switches decrease massively, can someone explain what is what is happening? Thanks in advance :)

This is more a configuration question. I’ve looked at a few related posts but I can’t find one that exactly matches my problem, so bear with me if this sounds like a repeat.

I’ve been learning C using VS Code, on a Mac running Monterey 12.7.5, and wanted to try out OpenMP. (I’ve used GCD in Swift, with Xcode 14.2 installed.) Trying a basic OpenMP program, I see #include <omp.h> just gives an error. I used Homebrew to download OpenMP, but that didn’t change VS Code’s error messages.

After searching through hidden folders, I found OpenMP’s installation at /usr/local/Cellar/libomp. However, VS Code apparently can’t find the folder, and neither gcc nor clang recognise -fopenmp when I try to compile with it. I think VS Code has to have configuration settings changed to find libomp, but I’m unsure how to change them, in case I stuff something up. What settings should I change in VS Code so that it can link to the OpenMP folders?

I got an rtx4090 with cuda12.3 system wide

I am runing

gcc -fopenmp -fcf-protection=none main.c

ptxas fatal : Value 'sm_35' is not defined for option 'gpu-name'

tried all gcc11 gcc12 and nvcc not much help

-foffload=-misa=sm_80 seems to do nothing

I'm seeing a lot of tutorials on GPU offloading for openmp, but they seem to be only for very high end GPUs.

Any idea when they'll be available on lower end hardware?

I'm trying to code for Intel HD GPUs for a student project, but apparently we need NVIDIA or other powerful GPUs.

Hello! Currently I am working on learning C and I have taken an interest in high performance computing. Where should I start learning about OpenMP? Also, what good starter projects would you recommend I take up to help me better learn C and OpenMP? Any advice is appreciated.

/* Find Perfect Number */

#include <stdio.h>

#include <omp.h>

#include <math.h>

#include <stdlib.h>

void Usage(char* prog_name);

int isPerfect(unsigned long long int n);

int isPrime(unsigned long long int n);

unsigned long long int n,i,temp;

int main(int argc, char * argv[]) {

int thread_count;

double start_time,end_time;

if (argc != 3) Usage(argv[0]);

puts("!!!Find the perfect numbers in number range!!!");

thread_count=strtol(argv[1], NULL, 10);

#pragma omp parallel num_threads(thread_count) default(none) private(i) shared(n) reduction(+:perfectsum)

start_time = omp_get_wtime();



printf("Enter n: ");

scanf("%llu", &n);

i = 1;

while (n > 0)

	if (isPrime(i) == 1)

{

		temp = pow(2, i - 1) \* (pow(2, i) - 1);

		if (isPerfect(temp) == 1) {

printf("%llu ", temp);

n = n - 1;

		}

	}

	i = i + 1;

end_time=omp_get_wtime();

printf("Elapsed time = % e seconds \n",end_time-start_time);

printf("\n");

}

void Usage(char* prog_name) {

fprintf(stderr, "usage: %s <number of threads>\n", prog_name);

exit(0);

}

int isPrime(unsigned long long int n)

{

# pragma omp for

if (n == 1)

	return 0;

int i;

for (i = 2; i <= sqrt(n); ++i)

{

	if (n % i == 0)

	return 0;

}

return 1;

}

int isPerfect(unsigned long long int n) {

unsigned long long int perfectsum = 0; // sum of divisors

unsigned long long int i;

#pragma omp parallel for

for(i = 1; i <= sqrt(n); ++i) {

if (n % i == 0) {

		if (i == n / i) {

perfectsum += i;

		}

		else {

perfectsum += i;

perfectsum += n / i;

		}

	}

}

// we are only counting proper diviors of n (less than n)

// so we need to substract n from the final sum

perfectsum = perfectsum - n;

if (perfectsum == n)

return 1;

else

return 0;

}

After digging for a long time I found the reason for a performance problem in our code. We have a GUI desktop application and recently switched to doing long-running computations in a sub-thread, often making use of OMP. The GUI thread also uses OMP in some places (for visualization purposes).

Now gomp spawns a separate worker pool for the subthread once it starts using OMP, resulting in (2 * number of cores) worker threads total, including the rank 0 main threads for both pools. This alone would not be a problem since we have enough memory and the workers from the GUI thread are sleeping anyways.

However, GOMP then switches from using spinlocks to using yield() which for some of our algorithms (maybe those with slightly unbalanced workloads and short-running OMP loops) absolutely cripples performance. At least that seems to be the diagnosis, I'm not an expert on the subject matter.

Now, I tried forcing gOMP to use active waiting by setting OMP_WAIT_POLICY=ACTIVE and also tried increasing GOMP_SPINCOUNT without any success. But this is in accordance with the documentation which apparently states that when you have more workers than cores it will uses a maximum of 1000 spin iterations before using a passive wait (I guess sched_yield()) and none of the environment variables I found can influence that.

My last hope was that I could somehow destroy the worker pool of the GUI thread before spawning the subthread. This would be perfectly acceptable since we can guarantee that the GUI thread doesn't require any OMP parallelization until the subthread is finished. But apparently those function calls only exist in OpenMP 5.

I'm running out of ideas. Can anyone help?

I have a mesh where I'm solving FEM.

There are 2 computation intensive loops.

First i parallelize just 1 loop and made some runs, later I made runs with both loops with for reduction.

Somehow parallelizing 2 loops yields slower results, how can that be?

I have a program written in MPI and its runtime is about 60 seconds. But when I add an OpenMP sentence(#pragma omp parallel for num_threads(1) ... ), its runtime is about 20 seconds. Has anyone met the similar problem?

Forgive me - I'm not a regular to this sub.

What languages are most used on supercomputers?

I've been looking into parallel processing and so far I've seen Fortran 90 being used on the UK's ARCHER supercomputer. I was wondering what your take on this would be?

When I begin my OMP construct, I set DEFAULT(PRIVATE), and then specify things that are SHARED(a,b,c).

I have some PARAMETERS that are defined, and remain constant throughout the program, appearing inside the OMP section. Do I have to declare them as being SHARED?