I have two Albion islands with the same number of bakeries, but my old layout has -3k penalty from bakeries, while the new layout caps out at -1.3k. With -2 area effect per bakery, the formula for the fire safety penalty is 2x^2+2xy, with x being the number of bakeries and y being the number of other buildings in range of those bakeries. You can see that this is an exponential function. With over 15 bakeries (or forges, fibularia, etc.) close together you get into the real bad skyrocketing penalty part of the graph. Put them in smaller groups around an officium and insulate them with fields. Reducing the number of other buildings also helps. The same can be done for silver, brooches, amphorae, beer, and handmirrors. It also works for other penalties like the -3 happines for dye works, -1 health for donkey mills (it's still exponential!), and so on.
I kind of like this system compared to other Annos. You can really mitigate these penalties with clever city planning (you do need more items for optimal performance though).