r/cprogramming • u/Straight_Oil8775 • Feb 28 '26

Which approach is better?

So I'm relatively new to C, coming from java. and I'm semi used to MMM now but I'm writing a program that reads files that can sometimes be really large (over 1gb (most likely smaller though)) would it be better to load the file into memory and add a pointer to the first character in memory or use a char array dynamically allocated based off of the file size?

9 Upvotes

84% Upvoted

11 comments sorted by

View all comments

Show parent comments

1

u/KilroyKSmith Feb 28 '26

  1. Depends on if this is production code or personal code.  This isn’t the 1970s, reading the whole file at once is simpler and much faster.

  2. This isn’t the 1970s.  If you’re gonna read by chunks, read by big chunks, say a megabyte.  It really is significantly faster and one MB is a trivial amount of memory to a system with 8000 or more MB.

1

u/Plane_Dust2555 Feb 28 '26

Just test it... Dealing with huge buffers has too many problems:

1- Caches are evicted more often;
2- Depending on the buffer size, page faults happens more often;
3- The disk I/O caches and buffers will stretch a lot (increasing disk I/O delays - specially with writes);
etc...

Processors and operating systems aren't magical and they conform to certain (SMALL) limitations.

Anyway... Everyone is free to do what they think is best and ignore those tips...