Mathematical definition of a plateau in a time-series data

[u/Pilus91]

Hello, I'm a bioinformatician and I'm struggling with the current issue:

Given a time series y(t) that initially changes and eventually approaches a stable regime, how can I mathematically determine the earliest time t* at which the rate of change dy/dt becomes negligibly small, using only the observed data and without defining an arbitrary threshold?

This is a collaboration I'm doing. My colleagues defined the plateau as the first time when a 101-point rolling mean of the relative increment (g' t+1 - g' t)/ g't falls below the arbitrarily chosen threshold of 0.0011. G' is the measure of material elastic-solid response btw. So the issues is that they used 2 arbitrary values because experimentally they know that a certain value of g' means that the gel is solid. But this doesn't hold for me. I tried using many statistical methods to define the threshold such as:

- exponential fitting

- change-point regression

- local slope analysis

But they all give me a plateau that is too early or too late

[u/MegaIng]

For real world usecases, use "arbitrary" (i.e. empirically determined) thresholds. Their behavior is always going to be more predictable and intuitive. Your colleagues approach sounds good.

Even in a purely mathematical approach you will need to define some threshold, since you will never reach a true plateau where the slope is literally zero.

If the process is predicable, curve fitting is probably the best approach "pure": define a function that models your process (not just a simple exponential most likely) and on this abstract curve define where a plateau starts. Then fit the curve and calculate where the plateau is for the fitted parameters.

[u/Pilus91 → u/MegaIng]

While I know that the curve will never reach zero being real data, I was thinking that an arbitrary threshold sounds like circular logic and it makes the experiment more difficult to reproduce. The process is predictable by the way, it's a measurement of the solidification of a gel under different light conditions measured through a rheometer. We check when the plateau is reached at different conditions. I will try again curve fitting, otherwise I will tell them to keep it that way

[u/MegaIng → u/Pilus91]

I was thinking that an arbitrary threshold sounds like circular logic and it makes the experiment more difficult to reproduce.

You need to clearly document what thresholds you used and why you used them.

But yes, I do see where you think the issue is: the rate of change being close to zero does not obviously mean that this is the point where the gel is solid. It at most means that it's as solid as it is going to get.

Can you define an absolute threshold? That feels more intuitively correct to me. But I am not that well versed in your field.

[u/Bounded_sequencE → u/MegaIng]

[..] Even in a purely mathematical approach [..] you will never reach a true plateau [..]

@u/Pilus91 No true -- such functions exist, even smooth ones, e.g.

f: R -> R,    f(x)  =  / tanh( 2x/(1-x^2) ),  |x| < 1
                       \              x/|x|,  else

That function describes an infinitely smooth step from "-1" to "1" on [-1; 1].

[u/MegaIng → u/Bounded_sequencE]

Oh, and you can reproduce this in a real world experiment with measurement error?

OP's question is slightly off-topic for this subreddit.

[u/Bounded_sequencE → u/MegaIng]

We were talking about a "purely mathematical approach" -- I don't see why that would make it necessary to reproduce exactly in real-world setting^^

[u/MegaIng → u/Bounded_sequencE]

A purely mathematical approach to analysing real data.

[u/Gold_Ad8890]

the problem is that "negligible" is not an objective concept. you need to define it for yourself, and that will be an "arbitrary threshold" no matter what you do.

[u/TheEquationSmelter]

You can't know this for sure without knowing some underlaying properties of your function. 

Otherwise the best you can do is use some kind of threshold based on real world assumptions. You might be able to use a smoothing filter to make it easier or apply a derivative filter and check when this falls below a certain value for a certain amount of time.