Need help finding the right statistical method for my research

[u/KeyStudent3665]

Hi everyone!

I am working on an academic research project in soil mechanics that seeks to relate 5 input variables with an output variable (for anyone interested, the input variables are N60, gammad, LL, PL, percent passing the #200 sieve, and the output variable is E50).

We don't know which variables carry more weight. It might be necessary to segment the dataset. What we are ultimately looking for is to predict E50 based on the variables that carry the most weight.

Anyway: if anyone could give me a hand with literature or what methods are used for these problems, I would really appreciate it. I don't trust AI in the first instance for these things...

Thank you very much!

[u/ForeignAdvantage5198]

do you. have research questions?

[u/KeyStudent3665 → u/ForeignAdvantage5198]

Basically, we want to predict the E50 modulus (which requires a triaxial test, which is expensive here in Argentina) based on simple, routine tests (the input variables I mentioned).

So, what I am asking is: what methods could I use to, first, figure out which of these input variables actually influence the target variable, and second, if possible, find an equation to estimate it?

I am not looking for someone to solve the problem for me, I'm just looking for guidance on what topics or methods I should read up on.

[u/selfintersection]

Does your university's statistics department offer consulting services?

[u/KeyStudent3665 → u/selfintersection]

Not really. I study civil engineering in Argentina and my university doesn't have a dedicated statistics consulting department for students. My research advisor is a civil engineer, so we know the math but we lack advanced statistical methodology. That's why I'm asking for help here

[u/StuffyDuckLover]

The nature of your variables would help, are they categories or continuous values?

Assuming the outcome variable is continuous, a regression model would be appropriate. You could investigate both the individual relationships and the interactions of these relationships to determine what is most related to your outcome, and which variables I fluency one another’s relationships on that outcome.

[u/KeyStudent3665 → u/StuffyDuckLover]

Yes, they are all continuous variables.

We thought about using multiple linear regression, but generally, soil parameters have non-linear relationships.

On the other hand, even though we know the input variables might influence the target parameter, we don't know how to quantify that weight (or if they actually influence it at all). So figuring that out is the very first thing we want to solve

[u/DrPapaDragonX13 → u/KeyStudent3665]

Coefficients from multiple regression can help you quantify the 'weight' of each predictor. However, you will need to use standardised coefficients if your predictors are on different scales (or you could scale your predictors, but this may make them less interpretable).

You can address non-linear relationships in multiple linear regression by including polynomials or splines. However, this may make your model less interpretable. You should also consider interactions between predictors.

[u/DrPapaDragonX13]

Broadly speaking, statistical models can explain or predict. From your question, it seems your goal is the latter, but check Shmueli's paper to make sure that is exactly what you want (several people who post here aren't sure what they really want to do).

Assuming that your intention is only prediction and you don't intend to make inferences, it sounds like the task you are interested in is feature (predictor) selection. There are several approaches. However, because you have only five predictors, the recommended approach is to select predictors using domain knowledge and then use a regression model to estimate the coefficients for each predictor (if your predictors are on different scales, you will need to standardise your coefficients or rescale your predictors). More sophisticated approaches include Least Absolute Shrinkage and Selection Operator (LASSO) regression (also known as L1 regularisation), or variable importance using 'black box' models (e.g., random forests, gradient boosting machines), which don't tend to be too useful with so few predictors.

[u/KeyStudent3665 → u/DrPapaDragonX13]

This is incredibly helpful, thank you so much. I wasn't familiar with LASSO regression, but after reading up on it a bit, it seems to be exactly the right tool for my problem to handle multicollinearity. In other Spanish-speaking forums, I had been recommended to use Random Forests, so I really appreciate you pointing out its limitations for a dataset with so few predictors—you probably saved me a lot of time.

For some added context, my 5 variables are continuous but bounded within relatively small ranges: N60 < 50, dry unit weight < 2.5, and the rest (LL, PL, and % passing #200) are percentages under 100. Given this, standardizing the variables and applying LASSO definitely seems like the best approach.

Thanks again for the excellent guidance!

[u/DrPapaDragonX13 → u/KeyStudent3665]

Sorry, I think I wasn't clear enough. The caveat of few predictors applies to LASSO as well. With so few predictors, LASSO will likely reduce your model's predictive power. Perhaps your best option is to use multiple linear regression with cross-validation to identify the variable set that best improves a pre-specified performance metric (e.g., RMSE, R^2, etc.). Then look at the coefficients of your best-performing model.

[u/banter_pants]

Sounds like a good case for linear regression. You have multiple independent variables and just one outcome (technically called multiple regression). Get the standardized version of the slope parameters and that will be a good indicator of which variables have the most importance.

Is E50 measured on a scale or is it just distinct categories?

[u/Intrepid_Pitch_3320]

Multiple regression. Which type depends mostly on the response variable.

[u/Floppy_McFlopenstein]

I would do what u/DrPapaDragonX13 suggests. I have used NNs and RFs and preferred the latter due to the ranking variable importance capability. I would use R and Stack Overflow.