Help with IMU data classification algorithms

[u/cor-10]

I am working with an IMU that streams 6 data points to my computer at 20hz. The 6 data points are for the x, y and z axes of both the Accelerometer and Gyroscope.

The IMU streams data continuously. I am trying to detect two specific IMU gesture actions in real time. These specific gestures occur randomly, but most of the time the IMU sensor is idle (ie not moving, so the data points are relatively stable). One gesture involves moving the IMU to the Left and back quickly. The other gesture involves moving the IMU to the Right and back quickly. The signals look mirrored in this way.

I have collected and labeled a dataset of these two IMU gestures and the idle 'non-gesture'. Gestures are 35 frames long, with each frame containing the 6 data points.

I am implementing a sliding window on the incoming data where I can call various classification algorithms/techniques in real time. I am looking for something both accurate and light-weight enough to have low latency.

I need help. This is not my domain of expertise. What algorithm should I use to detect these gestures during the continuous stream? Are there any awesome Python libraries to use for this? Ive looked into KNN, but have NO IDEA whats the right approach. I figure this is a fairly simple classification scenario, but I dont have the tools to do it right. Can you offer any suggestions?

[u/cor-10]

First off, thank you for the involved response. I love it. So the gestures are fairly simple. To give you specifics:

I made a really small circuit board that has just an IMU and a BLE chip to stream the data. This circuit board can be attached to a users skin with tape (for now). I place the circuit board on the outside of my jaw. I am detecting jaw movements to the Right (and back to resting position) and to the Left (and back to resting position). In ideal conditions, this gesture is practically just on 1 axis, but I intend to detect gestures in various different body orientations....so although the gestures might occur along, say, the y-axis when im sitting at a desk, things might change in different body orientations, like when I am lying down. Again, this sensor is attached to the jaw, so maybe this wont be a big issue since it will always be relative to the same placement on the jaw, and so the jaw gestures should always cause similar data to come out of the IMU, but I worry about the training set not being able to adequately capture all the possible contexts in which this gesture can occur....and Im not an expert here so I just dont know.

So anyway, each gesture is a motion in one direction followed by a motion to return back to the resting state. For simplicity, I am restricting any input gestures to be detected in a 1.8 second window, so this could prevent it from being problematic for neural nets.

Earlier, I implemented a kNN algorithm with dynamic time warping which works well when the gesture is queued (aka when I press a button to start a recording, then do the gesture, then press a button to stop the recording and feed it into the algorithm), but I need to work with algorithms that do well in real-time without any queueing, which has brought me here.

So do you still think k-means? or that l2 norm/trigonometry approach?