8.1 What is parameter estimation?

Over the next several sections we will examine aspects of parameter estimation, which can be generally stated as the following process:

Parameter estimation is the process of determining values of parameters \(\vec{\alpha}\) for a function \(f(\vec{x}, \vec{\alpha})\). Usually these parameters are determinined by minimizing the square difference between data \(\vec{y}\) and the output of the function \(f(\vec{x}, \vec{\alpha})\).

Example 8.1 The function \(f(x)=ax+b\) has parameters \(a\) and \(b\). In our notation above, \(\displaystyle \vec{\alpha} = [a \; b]^{T}\). Usually these parameters can be determined through a set of measurements \((\vec{x},\vec{y})\) (in other words a scatterplot).

Example 8.1 is an example of a linear parameter estimation problem. I did use matrix notation to denote the \(\vec{\alpha}\) - although I will say matrix notation might be a little more formal for our purposes to start. To solve this problem we can address it from several different mathematical areas: calculus (optimization), statistics (likelihood functions), and linear algebra (systems of linear equations).

In this section I will show you how to apply R to determine the unknown parameters and interpret the results. A few section later we will explore how to approach the parameter estimation problem with likelihood and cost functions. We will use R a lot in this section to make plots - so please visit Section 2 if you need some reminders on plotting in R.