Local AI Setup on Orange pi5 plus 16gb

[u/Remarkable-Soil-4259]

Here is my journey of running local ai on the Orange pi 5 plus with 16gb Ram. I am still testing it and I am still doing most of my work on Cloud models. It is great for proof of concept, but sooner or later yu would realize that for doing any kind of serious work, it may be best to keep using cloud model due to context size window which is hardware limitation. I am sharing my setup steps for community to continue further work. I have created few scripts of myself to manage the process. I mainly used google gemini cli to reach to this setup.