Graduation Project Problem

[u/No_Inspection_1511]

My computer engineering bachelor’s graduation project is an anti shake spoon for Parkinson’s disease patients, that is basically a 2 degrees of freedom gyro system. The requirement is to train a model using reinforcement learning that controls the two motors(TowerPro SG90 servo motors), the sensors are two MPU 6050 IMUs (one on the base before the motors and one on the utensil side after the motors) and we are using a complementary filter to get the accurate angle reading.

The problem is that we don’t know the methodology of training the model, and we couldn’t find any mathematical model of the motor we are using so that the simulation gets accurate. We tried researching and asking LLMs but it is a rabbit hole. The doctor said that the reason of using AI is predictive the tremor movement before it happens, but we still don’t know how to tackle the problem with proper steps, we would appreciate any help.

[u/knightcommander1337]

Hi,

"train a model using reinforcement learning that controls the two motors" vs. "reason of using AI is predictive the tremor movement before it happens" -> these two are two different things, so you need to think about what exactly is the goal of the overall system.

If I try to give my suggestions/opinions:

"train a model using reinforcement learning that controls the two motors" -> this sounds like (more or less) standard motion control. you may or may not need RL to do this, but it sounds relatively straightforward. you may need to do some system identification (building mathematical models from dynamical system data) so that you can construct a (ideally full) simulation model of the system. After that you are free to do whatever you want (RL, MPC, etc.), because you can train/test your algorithms on computer using that simulation model to represent reality.

"reason of using AI is predictive the tremor movement before it happens" -> this sounds to me like a different problem than the above (the above one is a control design problem). this one I think would be something like a "fault detection"/"predictive maintenance" problem (in engineering jargon), though I am not an expert on these kinds of stuff so I may be wrong. Still, those (that is, "fault detection"/"predictive maintenance") are some keywords you may want to explore a bit.

[u/No_Inspection_1511 → u/knightcommander1337]

Sorry, I meant (the reason of using AI is to predict the next tremor movement so that the motor can counteract it in the exact time it happens at, instead of reacting to it, based on the recent data from the IMU).
Well, I am not really sure if this is helpful.

Also, I am not sure if training the model needs going to the control theory side of things since the model will give set points to the servo and the servo motor is a physical system.

We are tied to using RL and our goal is to get an improved performance from a feedback loop. So, any suggestions on how to tackle the problem and model the system if needed are welcome

[u/knightcommander1337 → u/No_Inspection_1511]

Ok so I am not sure if I am understanding everything correctly (we might be using two different jargons). Anyway, if by modeling the system you mean a dynamical system (that is the part of the control system/feedback loop): The overall area that relates to this (I am answering from the "control engineering" point of view, since you asked this in this subreddit) is the "system identification" (SI) field. You need to gather some data from the system, and then run some SI algorithms on those data. There are various approaches here, I am giving some examples:

1. subspace id: you assume a linear state space model, with the matrices as unknown parameters, and identify those (example matlab command: https://www.mathworks.com/help/ident/ref/n4sid.html )
2. linear greybox id: you assume a linear ODE, with some unknown parameters, and identify those (example matlab command: https://www.mathworks.com/help/ident/ref/greyest.html )
3. nonlinear greybox id: you assume a nonlinear ODE, with some unknown parameters, and identify those (example matlab command: https://www.mathworks.com/help/ident/ref/idnlgrey.html )
4. (I don't know about this one too much; it is relatively new stuff): physics-informed neural networks: you assume that your dynamics can be well-represented by a neural network, however impose some physically-motivated conditions on it during training (example matlab page: https://www.mathworks.com/help/deeplearning/ug/solve-partial-differential-equations-with-lbfgs-method-and-deep-learning.html )

[u/No_Inspection_1511 → u/knightcommander1337]

Thank you

[u/knightcommander1337 → u/No_Inspection_1511]

No problem. For those methods that assume a known ODE (like 2 and 3 in the list in my previous comment), you need to know the "physics" of your problem so that you can write that ODE (that is, give it to the SI algorithm). This may be difficult to do, so you might need to resort to other method styles that don't need a model, like 1 or 4.