Confidence intervals on polynomial trends

[Luca Nori]

I'm honoured to open this section with a specific (may be too specific?) tech topic.

My problem is to fit regression curves to signals representing the soil strenght versus depth.

As you can see from the attached example in .ppt format, it would appear that, more often than not, polynomial curves best describe the typical fluctuations of the signal especially when their phases are not differing much.
Do you know an algorithm to build the confidence intervals of the mean (and confidence levels) around a polynomial fit such as the one illustrated in the example. If there is an easily implementable one.

Drawbacks of poly fit versus linear fit? Would you work out the problem another way?

[Luca Nori]

Sorry, maybe I should have added that the new Eurocodes regulating construction activities in Europe have simplified the rigorous risk assessment by the probabilistic reliability method (convolution of demand and capacity functions) introducing a so called semiprobabilistic limit states methods where low percentiles of the mean (when the spatial average of the property governs the soil rupture) is taken as a point estimate of the parameter.

So, instead of going thru the geostatistic process of detrending the signal, estimating autocorrelation, variance reduction and so on, I'll just take as representative the 5th percentile of the distribution of the mean of the properties which represent soil strenght, in example...

[William Huber]

Interesting questions.

Polynomial fits are usually a poor procedure except where (a) theory suggests a particular form or (b) there is an enormous amount of data relative to the degree of the polynomial. For example, it's not uncommon to use quadratic or third degree polynomials to "rubbersheet" one map layer in registering it to another. The data usually consist of numerous (dozens of) point pairs and the polynomials serve to approximate a projective transformation between two planetary coordinate systems.

The problem with polynomials is they behave very badly between sample points and outside the range of those points. This is well documented and discussed in most regression textbooks and in the numerical computing literature (www.nr.com, for example).

If you must do polynomial fitting, use a good multiple regression package. For example, a second degree polynomial fit of y as a function of x is achieved by regressing y on x and x^2. It will give you confidence intervals for the coefficients and prediction intervals for the values at unsampled locations.

I would indeed be inclined to work the problem some other way, but the technique of choice depends (among many things) on the kind of data, their quality, their quantity, any underlying theories or hypotheses about the behavior, the decisions to be made from the data, and the possible consequences of those decisions.