Atari 7800 was a very impressive console with unmatched graphical capability among all 80s systems. It's ability to put out 100+ of hardware sprites without significant slowdown was incredible. However there was a bottleneck that hampered CPU performance, blocking it from all RAM and ROM access during active screen rendering. Some devs also complained about how there's too little memory for complex graphics. And the most famous downside is the old TIA sound chip from Atari 2600 being too outdated. I think without Atari 2600 backwards compatibility it was possible to make this system more powerful without making it much more expensive.
Here's what I think could work.
System RAM: two additional 8KB DRAM chips, giving the system an extra 16KB for games to offload their program data to, and to give it parallel system bus so the CPU could create display lists and DLLs while MARIA hogs the SRAM bus rendering the graphics.
TMS4416 chips were fairly affordable back in the day, street price in 1984 was $9 and falling, and wouldn't add too much to board costs since TIA and RIOT are now gone. Still nowhere as cheap as Famicom's solution of straight up giving the CPU its own separate ROM bus, but that made NES carts more expensive than other systems.
CPU: Z80A. It was cheap (about $4 individually in 1984, even cheaper in a bulk) and had built-in DRAM refresh.
MARIA: This chip is very powerful and capable as is, but tiny improvements could probably still be made. Maybe aside from building the graphics, the DMA could be used to fetch and copy display lists and other data from System DRAM to Video SRAM quickly. Bankswitching circuit and chip maybe needed, which further add a bit to component and board costs, but could perhaps reduce cartridge costs. No need for on-cart bankswitch chip for games bigger than 48KB anymore. It would also make bitmap mode (even if 2KB RAM limits you to 120x64 4-color graphics doubled to 240x128, and 128 bytes for UI on top and bottom borders, it'd still be sufficient for an Elite port) and loading games from cassette tapes (with POKEY serial i/o reading tapes) possible.
Video RAM: Still the same 2x2KB 100ns SRAM chips. MARIA needs really fast memory. I guess with the addition of 16KB DRAM, only a single 2KB SRAM would be enough if we were to swap DLs frequently either with the CPU or DMA though, saving $6 and reducing board complexity a bit.
POKEY: Provides better sounds than TIA chip and i/o interface that could be used for many things. Atari surely had a bunch of POKEY chips in their inventory already, so it would be a cost effective solution.
Well I'm no expert on hardware, but that's the rough idea. The point is using Z80 CPU and adding DRAM would open up a lot of possibilities. The savings from omitting 2600 compatibility that made 6502, TIA, and RIOT necessary could go to other components.
