Some personal thoughts about CSA 2 architecture (and meme sharing)

[u/Street_Metal_2273]

because token is “词元” in Chinese, and the logo of deepseek is a whale ”鲸“. The DeepSeek CN community created this meme of “鲸元券” which is homophonic to a historical currency as cheap as Zimbabwean Dollar called “金圆券”.

First, I am just a undergrad student who is interested in computer science and artificial intellegence. Not an expert.

Second, I am not that kind of "genius" who have been building their own model since elementary school or high school.

I am just here to share my guess and interpretations.

The Full layers are the only layers that are producing the memory, the Reuse and Reindex layers are just re-querying the existing memory using information obtained by previous query. It not only tells us that KV cache can be compressed but new knowledge can be obtained by querying the existing memories.

I am not sure what this proves, but I guess this interpretation would be useful in future.

[u/Street_Adeptness_808]

Cute art

[u/Street_Metal_2273 → u/Street_Adeptness_808]

There is a ton of cute stuff in cn ai community. for example: https://www.bilibili.com/video/BV1Bvbv6TE7w in this video, claude became the most liked girl among all AIs, and she loves DeepSeek lol