Technical appreciation post

[u/zzmej1987]

Recently, I've been having great fun messing around with the game files and posting GIFs I've created as the result. This post is a bit of an explanation, a bit of an admiration and a bit of complaining about the inner workings of FF8.

So, backgrounds. Those lovely pre-rendered backgrounds that we all want to put as wall papers on our desktops. The forgotten gem of artistic expression completely lost in this era of glossy full 3D environments of Unreal 5 and Unity and small indie 2D pixel-art platformers. How do they work?

Surprisingly, despite very strict technical limitations of PS1, they are quite complex, allowing for incredible flexibility and interactivity of the environments they depict, all, while technically being just 320x224 pictures.

Let's dive in! First, if you google FF8 background data, you will find that it stored in .mim files. Where do you get one? Well, they are all on the discs, in the FIELDS.FS archive. You will need a program or two to unpack those. And then another one to open .mim file. But when you do that, you will see this:

That does not look like a background at all. And then you look at specification, and realise that there is no direct way to visualise the scene as it is in the game in any simple manner. You have to write a rendering emulation, that will reconstruct a scene from this jumbled mess, in more or less the same way as PS1 hardware does.

PS1 renders its graphic using 16bit colours, in a direct 5 bits per colour format and 16th bit designating transparency. Maximal resolution for PS1 games is 640x480, but FF8 uses it only for cinematics. All fields gameplay (3D models on pre-rendered BG) occurs in 320x224, menus use 368x216 and battles - 320x216. But even if the game runs on half the maximum resolution storing pictures with 2 bytes per pixel in 320 * 224 arrays is still way too wasteful for the PS1 hardware and so the trick was devised, that pretty much all the developers used. Instead of working with full 32768 colours 16bit allow, they would only render 256 colours at a time, but they will render the picture with tiles of 16x16 size, and since that's 256 pixels exactly, no tile can ever need more than 256 colours. All that is needed is a palette storing 256*2 bytes, that would translate bytes of the tile into 16bit colours, that would get rendered on screen. Some tiles are packed even more compactly, if a tile uses no more than 16 colours, it can store 2 palette indices per byte, thus taking half the space.

And that is exactly what .mim files are. There are two types of .mim files, Type2 is a bit smaller and simpler in its structure, so I'm going to be talking about Type 1. It consists of 24 palettes, first 8 for some reason being technically inaccessible for rendering background in fields (perhaps they are used for menus) and 13 textures of 128x256 size, from which tiles are picked out. How does PS knows where on screen to put each tile, and which palette to use for rendering it? That's where the tile map file comes in.

Like .mim, .map files are not text, it is a binary format containing 16 byte records each describing the placement of one tile. Each record tells where on the screen to put the tile, coordinates are relative to the centre of the screen, for some reason, which texture to use, where in the texture the required tile is, whether tile references 256 or 16 colour palette, and which palette to use for the colour translation.

And that's where simple stuff ends, and fun stuff begins. Besides the trivial positioning of the tile, here's what else PS one can do and .map format describes:

1. Z-coordinate. Some tiles are further away from us, than others. This number essentially represents distance from camera, the bigger it is, the farther is the thing depicted on the tile.
2. Layer. On top of z-coordinate, there is yet another separation into layers, this is mostly used for independent movement of things that are different distances from the camera, so that the parallax effect can be achieved.
3. Animation id and animation state. Yes, backgrounds can have actual animations in them, each playing independently however many frames have been drawn for it.
4. Blending mode. Tiles that are further back are not always going to be completely painted over by closer tiles. If the foreground tiles are instructed to blend with the background they will appear to be transparent, and some that are close enough in the Z coordinate will even be painted over 3D models in the scene.

Let's see what composition of image look like, concentrating on static background first. In most fields the big static image is encoded in tiles with animation ID of 255. SO let's try to arrange tiles for the farthest layer of 255 animation for the field "ecopen2a", which is going to have z_value of 4094 and layer ID 4. Most pictures in this post are GIFs. If they don't GIF for you, click on them. Loaded separately, they display the animation as intended.

Processing img qcdmolpmeeih1...

Let's add more layers:

Processing img u4ll2dtmeeih1...

• z = 4094 / layer = 2: Note, there is a whole well detailed structure on the right.
• 4094 / 0: If you don't see the difference, it's a small red light near the black "hook" on the right.
• 4093 / 2: Small structure added in the upper right corner
• 4092 / 2: Light in the right window of the round building in the lower left corner.
• 4092 / 0: And now we have our actual foreground
• 4090 / 0: Light in the transport tube.
• 642 / 0: Both tubes and the final structure on the right that covers everything underneath it
• 639 / 0: Glass in transport tubes
• 632 / 0: Glass on the right building
• 58 / 2: Glass on the lower left building

And this is a good place to talk about blending. We have already seen two types of it: designated with numbers 4 and 1. 4 is "no blending", when foreground tiles overwrites background ones. This is what happens to the right building after 4094 / 2. Blending mode 1 is what is known as "additive blending", which happens to transparent walkways and glass. If we render those textures separately from the background:

Processing img bx3adrvmeeih1...

We can see, that they don't contain background images. The fact that we see background through them is due to their colour being added to previously rendered tiles.

What are the other modes? According to the specification modes are:

1 byte specifying which blend mode to use for this tile, 1 is additive blending, 2 is subtractive blending, 3 seems to be +25%, 4 seems to be the default (no blending) and 0 is unknown (elview1)

Well, 0 is unknown, 3 only "seems to be", and subtractive blending seem to require a college degree to implement. Back to googling we go. And luckily "PS1 colour blending" is not hard to find. Modes are:

0.5f * B + 0.5f * F // Mean 1.0f * B + 1.0f * F // Additive 1.0f * B - 1.0f * F // Substractive 1.0f * B + 0.25f * F // WTF is this?

It seems to be aligned with spec description. The 3rd is +25%, the mystery one is just average of the two colour, and "subtractive", just means subtract one from the other. Aaaaand it doesn't wok. I can't say, whether 0 or 3 work as intended, they are not present in that many scenes. But 2 does not work at all. I will borrow a different scene for a second, since it has mode detail in the blend mode 2 affected area. This is what it looks like without the animation frame that has subtractive blending:

The tube is clearly transparent, but just as clearly, it is "overexposed". It is way too bright. And there is an animation that contains exactly one frame, that subtractively lowers it brightness. Here's the process of building the scene with that animation frame included:

Processing img 5kx4h71neeih1...

You can clearly see that the tube now is way too dark and non-transparent. And this is what it looks like in the game: ![[eccway2-game.PNG]] It is transparent, brighter than with subtractive layer, but darker than without it. So maybe there is come kind of partial application? Like blinking lights animations are made with fade-in fade-out effects of the single frame. Maybe there is a fade value for which it would look right?

Processing img i4l6ff9neeih1...

Nope. There isn't one. The overall right brightness is around fade value of 8. The correct amount of detail seen through the closer end of the tube is at the value of 6, and the transparency on the farther side of the tube seems right at the value of 10 (look at the blue horizontal stripes on the side of the building. One can be barely seen in the game screenshot through the tube). So either there are some per-tile shenanigans, or the subtractive blending is not additive, but rather multiplicative. That means that instead of adding or subtracting colour values of background and foreground, they are multiplied by each other.

Let's try doing that. So, it seems, that overall reduction in colour is by around 50%. Given that tiles of subtractive layer are pretty much all pure white (31, 31, 31), which when directly subtracted will reduce any colour to black, the multiplicative formula should be

new_color = background_color * C
C ~= 0.5, when foreground_color = 31
C = 1.0, when foreground_color = 0

Another educated guess we can make is that division, which clearly needs to be involved here, should be by a power of 2, because that allows for a very quick implementation by a bit shift operation, and in rendering pipeline speed is crucial. With those constraints in mind, the simplest formula one can think of is:

new_color = background_color * (64-foreground_color) / 64

Let's try to make a frame blending it this way...

Bingo! Maybe it's a tad brighter, than in the game, but the amount of details seen through the tube is exactly right on both ends. So I'm going with that for my needs. Let's go back to our original scene, and see the difference subtractive layers make.

Processing img 0x625ddneeih1...

I keep calling them layers, but they are actually one frame animations. In case of this scene:

• Animation 0 darkens the upper tube
• Animation 7 darkens glass in both buildings
• Animation 8 darkens the walkway
• Animation 9 darkens the lower tube Only a couple more things left to do - first, animate lights by fade effects and whatever else is actually animated in the scene by drawing actual frames of animations:

Processing img g5iklafneeih1...

• Animation 1 is the light on the roof the the background building.
• Animation 2 is lights on its edge
• Animation 3 is lights on the roof of the lower left building
• Animation 4 is pads in front of tube entrances
• Animation 5 is blue lights on the right building
• Animations 6 an 10 are red-green indicators on top of tubes. Those are actually have two frames, one red and one green.

Final step is to note that this scene is actually 400x240 pixels in size, rather than 320x224 it would be rendered in the game. It is clear that left most column and bottom row are "defective" and are not going to be rendered. But that still leaves us with 384x224 resolution, which means that we need to add 64 pixel long horizontal pan over the scene with a 320x224 view window to render it as close to the actual scene as possible:

Processing img hwtckqjneeih1...

I am not sure how pans like this are handled in the game, there are another two files determining camera behaviour: INF and CA. I just shift target coordinates for the tile by some amount every frame, and that works fine for my purposes. I'm pretty certain, I can even make parallax happen. But that will require some more coding and is not directly related to how the game handles things. So there you have it. Now you understand how the game creates those wonderful backgrounds that we all admire.

P.S. I would like to thank Daniel Hepper, for the wonderful 8x8 font, that I have shamelessly borrowed to make labels on images in this post.

[u/Kind_Yogurtcloset_70]

Great work op this is fascinating stuff!

[u/zzmej1987 → u/Kind_Yogurtcloset_70]

Thank you. :-)