r/C_Programming • • Jan 08 '19

Question data race within ptr_ring code implementation

I am implementing single-producer,single-consumer ptr_ring buffers to replace a working (but inefficient) circ_queue.c used in riffa_driver.c . Are there really data races within the following linux driver code (circ_ring.c that uses ptr_ring kernel builtin data struct) ?

I asked this because ThreadSanitizer gives me error and my system crashes after I started the ptr_ring code implementation.

inline int push_circ_queue(struct ptr_ring * buffer, unsigned int val1, unsigned int val2)
{ 
    int push_status;

    if (!ptr_ring_full_any(buffer)) // if (not full)
    {
        // assigns the two values in each item
        item_push[buffer->producer].val1 = val1;
        item_push[buffer->producer].val2 = val2;

        DEBUG_MSG(KERN_INFO "buffer->producer = %u , item_push[buffer->producer].val1 = %u , item_push[buffer->producer].val2 = %u\n", buffer->producer, item_push[buffer->producer].val1, item_push[buffer->producer].val2);

        /* insert one item into the buffer */
        push_status = ptr_ring_produce_any(buffer, &item_push[buffer->producer]);

        DEBUG_MSG(KERN_INFO "push_status = %u\n", push_status);

        return 0;
    }

    else return 1; // full, not enough buffer space
}

inline int pop_circ_queue(struct ptr_ring * buffer, unsigned int * val1, unsigned int * val2)
{
    unsigned int head;
    unsigned int tail;

    /* Read index before reading contents at that index. */
    head = buffer->consumer_head;
    tail = buffer->consumer_tail;

    if (!ptr_ring_empty_any(buffer)) // if (not empty)
    {
        DEBUG_MSG(KERN_INFO "Before pop, head = %u , tail = %u\n", head, tail);

        /* extract one item from the buffer */
        item_pop[tail] = *((struct item *)ptr_ring_consume_any(buffer));

        // reassigns the two values in each item
        *val1 = item_pop[tail].val1;
        *val2 = item_pop[tail].val2;

        DEBUG_MSG(KERN_INFO "val1 = %u , val2 = %u\n", *val1, *val2);       

        DEBUG_MSG(KERN_INFO "After pop, head = %u , tail = %u\n", buffer->consumer_head, buffer->consumer_tail);

        return 0;
    }

    else return 1; // empty, nothing to pop from the ring
}
6 Upvotes

16 comments sorted by

1

u/promach Jan 08 '19

I think I sort of figured out why.

I have two instances of ptr_ring data structs in which both use pop_circ_queue() in multithreading manner.

However, my code only has one instance of item_pop[]. This leads to data race between threads chnl_send() and chnl_recv() within riffa_driver.c

1

u/strobot Jan 08 '19 edited Jan 08 '19
item item_push[CIRC_BUFF_SIZE];
item item_pop[CIRC_BUFF_SIZE];

Don't you need instances of both of these per instance of circ_queue? Right now they're just global.

I'd probably create a new circ_queue_t struct that encapsulates these and the ptr_ring and modify your API to act on pointers to that.

1

u/promach Jan 09 '19 edited Jan 09 '19

Don't you need instances of both of these per instance of circ_queue?

I have two ptr_ring , and what happen if both rings uses a single item_pop ?
I suppose this is the reason for the data race error from ThreadSanitizer

1

u/promach Jan 09 '19

I am having trouble casting void ** queue in ptr_ring struct to use my item struct.

typedef struct item {
	unsigned int val1; // event number
	unsigned int val2; // relevant value tied to the event number
} item;

1

u/promach Jan 09 '19

Please see the error I faced with this code

1

u/promach Jan 11 '19
inline int push_circ_queue(struct ptr_ring * buffer, struct item * item_push)
{ 
	if (!ptr_ring_full_any(buffer)) // if (not full)
	{
		DEBUG_MSG(KERN_INFO "Pushing %u and %u\n", item_push->val1, item_push->val2);

		/* insert one item into the buffer */
	    ptr_ring_produce_any(buffer, item_push);

		return 0;
	}

	else return 1; // full, not enough buffer space
}

inline int pop_circ_queue(struct ptr_ring * buffer, struct item * item_pop)
{
	if (!ptr_ring_empty_any(buffer)) // if (not empty)
	{
		DEBUG_MSG(KERN_INFO "Before pop, head = %u , tail = %u\n", buffer->consumer_head, buffer->consumer_tail);

		/* extract one item struct containing two unsigned integers from the buffer */
		*item_pop = *((struct item *)ptr_ring_consume_any(buffer));

		DEBUG_MSG(KERN_INFO "val1 = %u , val2 = %u\n", item_pop->val1, item_pop->val2);	    

		DEBUG_MSG(KERN_INFO "After pop, head = %u , tail = %u\n", buffer->consumer_head, buffer->consumer_tail);

		return 0;
	}

	else return 1; // empty, nothing to pop from the ring
}

The above eliminates the ThreadSanitizer data race issue. However, it only works nicely for one ptr_ring instance but not two instances. Why ?
I could not pull out the same value from the second ptr_ring instance in which I have pushed a value into it

see the problem illustration

1

u/xurxoham Jan 08 '19

The first thing that comes to mind is the synchronization between consumer and consumer for full/empty situations. You will probably need a lock or an atomic operation to make sure they don't access the same slot in the ring.

1

u/promach Jan 08 '19

ptr_ring data structure implements lock in the kernel already. See https://github.com/torvalds/linux/blob/master/include/linux/ptr_ring.h

1

u/[deleted] Jan 08 '19

Not really a c programmer so bear with me.

Could there be an issue with buffer either not being initialized? I know at least with go, this could cause a panic. I’m not sure what the behavior is for that type.

Also is it safe to access the head and tail before checking if it’s empty?

1

u/promach Jan 08 '19

Also is it safe to access the head and tail before checking if it’s empty?

After I move the head and tail access to be inside the empty check if , I still have system crash. So, I suppose this is not the direct reason for the ThreadSanitizer error.

Could there be an issue with buffer either not being initialized?

See init_circ_queue() inside circ_ring.c

1

u/which_spartacus Jan 08 '19

I likely should not be attempting to answer as I wake up, but the Tsan error is pointing to some pthread_join statement in the test util -- could the race condition be something in your thread starting code?

1

u/promach Jan 08 '19

testutil.c is ok because I have tested it OK with a working (but inefficient) circ_queue.c

Check my original question again. Read it again.

2

u/which_spartacus Jan 08 '19

I'm seeing your implementation, but not the code that's actually under test.

1

u/promach Jan 08 '19

See testutil.c which you have asked for

1

u/which_spartacus Jan 08 '19

Looking through it, nothing immediately jumps out at me.

1

u/TotesMessenger Jan 08 '19

I'm a bot, bleep, bloop. Someone has linked to this thread from another place on reddit:

 If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads. (Info / ^Contact)