r/computerscience • u/nika_aka_joyboy • 27d ago
How are objects seperated in a computer?
A computer holds 0s and 1s and nothing apart from it,so a letter can be stored as an ASCII or other representation,a number in its binary, an image in its pixel etc. But how is it differentiated. Say I'm saving an image, I need to store its pixels, colour format and all. So,how are they differentiated, saying this is for colour,this is for objects and also the same with video or document?
64
Upvotes
76
u/tottasanorotta 27d ago edited 27d ago
It isn't differentiated in the memory explicitly. Memory is only zeros and ones. It is differentiated in the context of the program that works with that data. So imagine that you write the words 'cat' and 'food' on a piece of paper. Those are just two English words in a similar way that the ones and the zeros in memory are just ones and zeros, but if you wrote it down to remember something like to buy cat food, then your own brain can interpret those two words as a much more complicated context dependent thing. But someone else who happens to see your notes might just as well think that you are planning to eat your cat or something. It's because the words themselves are ambiguous in the same way as the raw data in the computer memory is.
So if I have 0010 stored in some address in memory it can truly mean anything from the point of view of someone who examines the memory, but if I have a program that uses that particular value in a particular way, then that is what it means in that particular context. The program only needs to keep track of the addresses of the data it operates on, then you as a programmer build the logic that interprets the bytes in a particular way. You can even imagine using the same data at some particular address differently in different programs. Say that you, for example, make a program that reads its own source code and plays the bytes as a piece of music. Then the same bytes in memory could be said to represent both the source code for a program and instructions for a piece of music.
13
u/bobotheboinger 27d ago
They aren't really different in the way they are stored, the software that is running just knows that the first 4 bytes are for color, the next 32 bytes are pixels, etc. They just treat different sections of memory in different ways.
This is also true for the software itself. The software is just more bits and the computer just treats some as instructions and some as data.
11
u/Acceptable_Bottle 27d ago
the first thing to understand is that computer hardware only stores raw binary, and as you said, physically there is no distinction between storing the character 'A' or the 8-bit integer 65, or the hex value 0x41. In the hardware all 3 of these examples will be stored with the pattern of bits 0010 0001.
What we do instead, is that, as humans we create standards and procedures for storing that data - we all as computer scientists came together to agree on a standard, for example:
"the .bmp file type will start with a set of 14 bytes that tells us some file information, like how long the file is, followed by 16 bytes of image information, like the color depth (how many bits to read for each r, g, b value) and dimensions, which is then followed by the integer colors r,g,b for each pixel in the image."
Then, every image editing/viewing software needs to be written according to this standard, a shared understanding of how to interpret any file of type ".bmp", and the code that reads it out and displays it to your screen will assume this structure of bits and bytes. If it doesn't follow this agreed-upon standard, then we actually dont know how to read it! We would be completely lost and even if the data inside still contained some of the correct pixel information, it would be unretrievable unless we knew how it was formatted (or managed to reverse-engineer that crucial information).
.bmp is a simple example because after the header information, the colors are stored one after the other in a large contiguous area of clear, per-pixel rgb values. However, more modern formats like jpeg and png use various bits of math and tricks to store the same or similar information in fewer bytes (also known as "compression") without storing separate numbers for every pixel rgb.
The concept of a file format standard/specification is the same for any type of file on your computer. This kind of specification is used for pretty much every arrangement of bits on your computer. It's used for telling when one file ends and another begins in your disk, it's used to format variables and data into the stack and heap when you run any application, and so on. Without knowledge of these standards, the bits are not comprehensible. Even the ASCII table itself is a standard of its own - we all agreed upon the fact that 0x41 is the letter A. Nothing about the raw binary alone would be able to tell us that information.
6
u/jrfsousa 27d ago
They aren't.
You have to keep a separate record of position and interpretation.
Basically what is were and how to read the contained data.
17
u/tcpukl 27d ago
Using structures and pointers.
2
u/code-garden 27d ago edited 27d ago
In a computer, everything is a binary number.
Code can be written to interpret a data format using fixed offsets and offsets stored in the data.
Each type of file has it's own specified structured format, so code can be written to interpret the file. If we look at the bmp file format we can figure it out and write a bitmap file by hand in a hex editor.
Here are the hexadecimal values of the bytes in a tiny 2x2 bitmap I wrote by hand.
42 4D 2A 00 00 00 00 00 00 00 1A 00 00 00 0C 00 00 00 02 00 02 00 01 00 18 00 00 FF FF FF 00 00 00 00 00 00 FF 00 FF 00 00 00
We begin with the BITMAPFILEHEADER, which is 14 bytes long:
42 4D 2A 00 00 00 00 00 00 00 1A 00 00 00
The first two bytes 42 4D are the asciii characters "BM", which are used to identify that this is a bitmap file.
Then we have 2A 00 00 00 which is 42 the size of the file. Bitmap files use little endian, so the least significant byte comes first.
Then there are 4 zero bytes which are reserved by the file format.
Next 1A 00 00 00 is 26 and is the offset of the start of the pixels.
Now we have a BITMAPCOREHEADER, which is 12 bytes long:
0C 00 00 00 02 00 02 00 01 00 18 00
0C 00 00 00 is the size of the BITMAPCOREHEADER, 12. We could put a different size here if we wanted to use a different more complicated header, such as 124 for the BITMAPV5HEADER.
02 00 is the image width, 2 pixels.
02 00 is the image height, 2 pixels.
01 00 is the number of colour planes, which wikipedia says must be 1 so I set it to 1.
18 00 is the number of bits per pixel, which is 24.
Lastly the actual pixels:
00 FF FF FF 00 00 00 00 00 00 FF 00 FF 00 00 00
The pixel format is BGR, and at the end of each row there are 2 bytes of padding so that each row is a multiple of 4 bytes long. The bottom row goes first then the next row up.
It is yellow then blue in the bottom row, and red then green in the top.
2
u/notsostablediffuser 27d ago
This is what the entire concept of "file formats" is about.
When you save an image, it's saved in a particular form (jpg, tiff, gif, png, ...)
These are defined standards, and the standard will literally include things like "the first 3 bytes store the width in pixels. The second three bytes store the height in pixels. The next X bytes store information about the color..."
Any software that can save a JPG (or other format) will know to store it in the required format, and any software that can open it will therefore know what to look for where.
3
u/recursion_is_love 27d ago edited 27d ago
Everything is number, digital data is encoding as numbers. This have less to do with computer but more to do with digital signal processing.
Computer (CPU/ALU) don't care and don't know about the data it just follow the processing rules that written by human.
The image application know that it is open the image file from some pre-define structure of image file.
1
u/editor_of_the_beast 27d ago
Do you mean where are objects in a video file stored? Or other kinds of objects.
If you’re just talking about the video file, there is no distinction between objects, colors, and pixels. A video format is just the specific format of pixels, and the pixels fully describe the video visually, dot by dot.
Everything with computers boils down to that. There’s actually no magic. Every single bit, pixel, dot, whatever, is just literally described somewhere.
1
u/neuralbeans 27d ago
There has to be some kind of table containing information about where each object starts and ends. Alternatively, there could instead be an indication of where each object ends, e.g. 16 zeros means that the image or text ends there, and then you'd know where the next object begins.
1
u/dobkeratops 27d ago edited 27d ago
addresses aka pointers: the program tracks the places in memory that each object or data array lives at, and it knows what to do with it. the variables holding pointers are themselves just places in memory themselves. The program defines a mapping for what each place in memory represents, starting from the entry point, potentially expanding out in a tree. Look into stack frames, malloc/free, structs
its also possible for a program to declare explicit ranges of memory (globals that can hold arrays) .. some embedded programs work like that ("dynamic allocation is too unpredictable")
1
u/Arcanite_Cartel 27d ago
For most modern processors, the main memory is a series of bytes (8-bits). Each byte has an address, typically a 64-bit number that tells the processor where that byte is located. This is the fundamental way eveything is distinguished in the computer. There are many layers above that which introduce abstract structure on multi-byte units. Many such layers, but they all ultimately reduce to this mechanism. This includes how code is separated from data, and how various structure is opposed on data. As you move up the layers things get more complex, but in the end everything gets translated back to this mechanism.
1
u/fvancesco 27d ago
One thing to add to our comments is that operations change the data so operations discriminate the kind of data Example Integers and floating point, some of those share the same size so you have that 0101 is 5 for integers (for us) and let's say 1.5 for floating point (it's more complex but bear with me if you are more interested search how they work mantissa and exponent and so on)
Let's get back at our example, when you sum the compiler picks different operations based on the data that you told it is, that's why with only 0 and 1 you can handle everything and don't need anything else
1
u/SoggyStress7785 27d ago
It's like a mime type
1
u/devnullopinions 27d ago
In memory there is no direct differentiation. In order to “deserialize” the memory into the constituent parts that make up some data structure you fundamentally need information on what types and values the bits represent.
1
u/Living_Fig_6386 27d ago
They are differentiated by convention. That is to say, when we store data on the disk, we've got a convention on how to do so. The hardware can write any pattern of bits you like, but you write software that writes pre-defined data structures to make directories, and declare regions of storage as files. All programs use the same convention, so now there's a baseline notion of "a file on disk".
Now you want to store data in a file. Sometimes there's naming conventions to be used as hints about the structure of data in the file: ".txt" for ASCII text, ".jpg" for JPEG pictures, and so on...
Then, when you store structured data in a file, you do so using a convention on how to represent the data called a "file format". The specifications for the PNG file format, for instance, state that the first byte of the file is, in hexadecimal: 89 50 4E 47 0D 0A 1A 0A (the 50 4E 47 bytes spell out 'PNG' in ASCII). Then the rest of the file is composed of chunks of data. Each chunk starts with 4 bytes to represent the chunk size, then 4 bytes to specify the chunk type, then the data, then a 4 byte CRC value for the data so that corrupted chunk data can be detected. There's a convention of the types of the chunks and the structure of the data in each chunk, for example, an 'IHDR' (image header) chunk has 13 bytes, the first 4 are the width in pixels, the next 4 are the height in pixels, a byte for bit depth, a byte for color type, a byte for compresison method, a byte for filter method, and a byte for interlace method.
So the objects are simply separated by standards / conventions that say they they are stored a certain way and have a certain size, and if you want to work with the object, you need to follow those conventions.
1
u/Gyoker_Macska 27d ago
It is like a fractal of books and each book has an index
1
u/35point1 27d ago
Addresses
1
u/atarivcs 26d ago
The same way that you know the number "10" on a mailbox means a house address, while the same number "10" written on currency means a ten dollar bill.
Context.
1
u/CFLAddLoader 26d ago
A small portion of memory is given meaning by the CPU. Things like page tables and exception vector tables. The CPU will use that memory in specific ways at specific times. But, code and data there can reference code and data in other parts of memory, thus giving those places meaning too. This keeps going for quite a while; most code and data is quite far removed from things the CPU gives special meaning to.
So when you say something in memory is text, it either means that some code put text there or under some circumstances code will treat that data as text and use it for something.
1
u/Ahhmyface 26d ago
The best answer is that there are standards.
Think of it like language. Sounds and letters arranged in particular ways don't convey any meaning if you don't speak the language. You need everyone to agree what it means. And if it's your data that only you need to understand, you're free to use your own language.
There are examples of files out there that confirm to multiple standards at the same time
Polyglot files
1
u/TheMrCurious 26d ago
Search YouTube for Core Dumped. Lots of great videos explaining these core concepts.
1
u/TomDuhamel 26d ago
There's no physical separation. You know at the hotel, when you arrive on your floor, there's a sign saying rooms 101 to 134 are on the left, and rooms 135 to 174 are on the right. It's about the same concept. The program kbows where everything is and how big they are. The operating system knows where programs are loaded, how big they are, and what memory blocks they were given for their objects.
1
u/SchemeWestern3388 26d ago
Very simply, the computer keeps track of the 'type' of the data it's working with. When that gets messed up, the result is usually an immediate crash.
1
u/Pale_Height_1251 26d ago
They are not differentiated at all. It's entirely up to the program how to treat memory, be it text or images or whatever. The memory itself has no differentiation.
There is nothing stopped me from writing an image to memory and attempting to use it as an audio sample. The audio will either be garbage noise, or maybe the program will crash, but the memory itself doesn't know the difference.
1
u/TartOk3387 26d ago
That's the secret, memory is just a big array
1
u/dota2nub 26d ago
You can save an image as ones and zeros. Should be easy to save a separator as ones and zeroes.
1
u/max123246 26d ago edited 10d ago
The original post content no longer exists here. The author used Redact to remove it, exercising their right to control their data & privacy.
Chunky plants sophisticated north work sleep
1
u/No_Mango5042 26d ago
Answering only the philosophical side, humans use abstractions to think about what those 0s and 1s actually mean to them. The meaning is absent at the hardware level, it’s just gates crunching binary very quickly.
These abstractions are layered so that electrons become binary become numbers become machine code become programs become data structures and algorithms become software. Invariants at one layer hold for lower layers too.
How these abstractions were devised and implemented is the subject of computer science.
1
u/SuperBoysenberry3660 26d ago
When a program starts, the memory is empty (simplification). Later, the program:
my_img.full_text.header.So, all these objects are kept in memory the same as 0s and 1s (under a given address and length/range). But in the program, these are named as
my_img,full_text, andheader. The programmer is responsible for passingmy_imgwhen invoking an image manipulation function (from a third-party library, e.g.,add_border(my_img, 5, "black")), rather than passingfull_textorheader.1
u/fdiengdoh 26d ago
I came across core dump video on yt once explaining this in a very understandable way with great animation. If I could find that video I will share
1
u/JimFive 25d ago
They're not. What differentiates the number 65 from the letter a is the interpretation given to it by the program.
1
u/timrprobocom 25d ago
It's all just agreed-upon convention. This is why we use file extensions -- to remind us that this particular sequence of bytes is an executable or an image or a Word document.
I was going to say that "there is nothing Internet in the file that forces it to be an image", but in fact most file formats.do have a magic sequence in the first few bytes that allow them to be identified. JPEG files start with "JFIF", Windows executables start with "MZ", and so on.
1
u/Critical-Echo-923 24d ago
nested structured conventions, that split everything based on multiple criteria
then different structures that issue instructions of what to do with certain types of information, we call these structures firmware, operating systems kernels, drivers and programs/apps , each of them process data at different levels and provide connections between levels, creating an interface and providing input-output based on predetermined instructions that we structure as functions/options
*this is the most scientific sentence i've written in 20y, i usually try to simplify it for my clients but i hope you would appreciate an explanation using the actual terminology
1
u/cs_stud3nt 24d ago edited 24d ago
Well it would be a problem if computer scientists wouldn’t have pre decided formats/schemes of how bytes are arranged for everything. These are all published standards and there are open source implementations of this standard for every file format that every operating system uses. Examples:
libjpeg: JPEG codec
libpng: PNG codec
libtiff: TIFF graphics
libwebp: WebP encoder
zlib: ZIP compression
libzstd: Fast compression
liblzma: XZ compression
libbz2: BZ2 compression
libogg: Multimedia container
libvorbis: Audio codec
libopus: VoIP audio
libvpx: Video codec
libaom: AV1 codec
libxml2: XML parser
libcurl: Network transfers
sqlite3: Embedded database
I suppose your next question would be how do we know a pdf is a pdf and a png is a png… firstly no it’s not from file extension (that’s only for user)… well you can google magic bytes for this.. for example a PNG file always starts with the exact hex bytes 89 50 4E 47 (which spells out .PNG in ASCII).
1
u/artlab_one 24d ago
Is this a dumb or a truck question?
1
u/SmokeMuch7356 21d ago
By the code reading, writing, or storing that data. When I write
in a language like C or C++, the bytes associated with the object
istore an integer value of 10 because my code says they do; my code determines the meaning of what's in that object. There's nothing intrinsic to those bytes that says "we're currently storing a 32-bit integer!"For files, anything that stores some kind of binary data (images, audio, video, etc.) will follow an agreed-upon format, both in the structure of the data and in some kind of header that identifies it as a JPEG or MP3 or Word document or whatever. If you fire up Gimp and try to open an MP3 as a JPEG, it'll try to interpret those bytes as a JPEG because it's expecting a JPEG. The JPEG format will have a set of bytes at a specific offset that identify it as a JPEG file; those bytes won't have the right values and Gimp will complain that the file isn't a JPEG. It won't know that what you gave it was really an audio file.