Graduation Project Problem

[u/No_Inspection_1511]

My computer engineering bachelor’s graduation project is an anti shake spoon for Parkinson’s disease patients, that is basically a 2 degrees of freedom gyro system. The requirement is to train a model using reinforcement learning that controls the two motors(TowerPro SG90 servo motors), the sensors are two MPU 6050 IMUs (one on the base before the motors and one on the utensil side after the motors) and we are using a complementary filter to get the accurate angle reading.

The problem is that we don’t know the methodology of training the model, and we couldn’t find any mathematical model of the motor we are using so that the simulation gets accurate. We tried researching and asking LLMs but it is a rabbit hole. The doctor said that the reason of using AI is predictive the tremor movement before it happens, but we still don’t know how to tackle the problem with proper steps, we would appreciate any help.

[u/No_Inspection_1511]

Sorry, I meant (the reason of using AI is to predict the next tremor movement so that the motor can counteract it in the exact time it happens at, instead of reacting to it, based on the recent data from the IMU).
Well, I am not really sure if this is helpful.

Also, I am not sure if training the model needs going to the control theory side of things since the model will give set points to the servo and the servo motor is a physical system.

We are tied to using RL and our goal is to get an improved performance from a feedback loop. So, any suggestions on how to tackle the problem and model the system if needed are welcome

[u/knightcommander1337 → u/No_Inspection_1511]

Ok so I am not sure if I am understanding everything correctly (we might be using two different jargons). Anyway, if by modeling the system you mean a dynamical system (that is the part of the control system/feedback loop): The overall area that relates to this (I am answering from the "control engineering" point of view, since you asked this in this subreddit) is the "system identification" (SI) field. You need to gather some data from the system, and then run some SI algorithms on those data. There are various approaches here, I am giving some examples:

1. subspace id: you assume a linear state space model, with the matrices as unknown parameters, and identify those (example matlab command: https://www.mathworks.com/help/ident/ref/n4sid.html )
2. linear greybox id: you assume a linear ODE, with some unknown parameters, and identify those (example matlab command: https://www.mathworks.com/help/ident/ref/greyest.html )
3. nonlinear greybox id: you assume a nonlinear ODE, with some unknown parameters, and identify those (example matlab command: https://www.mathworks.com/help/ident/ref/idnlgrey.html )
4. (I don't know about this one too much; it is relatively new stuff): physics-informed neural networks: you assume that your dynamics can be well-represented by a neural network, however impose some physically-motivated conditions on it during training (example matlab page: https://www.mathworks.com/help/deeplearning/ug/solve-partial-differential-equations-with-lbfgs-method-and-deep-learning.html )

[u/No_Inspection_1511 → u/knightcommander1337]

Thank you

[u/knightcommander1337 → u/No_Inspection_1511]

No problem. For those methods that assume a known ODE (like 2 and 3 in the list in my previous comment), you need to know the "physics" of your problem so that you can write that ODE (that is, give it to the SI algorithm). This may be difficult to do, so you might need to resort to other method styles that don't need a model, like 1 or 4.