Convex Optimization 2026, 30 - Experiment Design

To estimate a vector x ∈ ℝⁿ from measurements or experiments with independent Gaussian measurement noise w_i with zero mean and unit variance is y_i = a_i^Tx + w_i. a_i spans ℝⁿ. The maximum likelihood estimate of x is given by the least-squares solution. The associated estimation error has zero mean and covariance matrix. The matrix E characterizes the accuracy of the estimation for the informativeness (p. 384). Experiment design chooses the vectors a_i from the possible choices so that the error covariance E is small. This is usually an optimization problem. The basic experiment design problem minimizes the error covariance matrix E to (∑^p_{j = 1}m_jv_jv_j^T)^-1 (Eq. 7.23). This may prove difficult from a combinatorics perspective, depending on the size of m. Instead, the error covariance could be expressed while normed by 1/m (7.5.1). The vector optimization problem of experiment-design can be limited to a positive semidefinite cone via scalarization. D-optimal design minimizes the determinant of E, which corresponds to minimizing the volume of the resulting confidence ellipsoid. It ignores the 1/m factor in favor of taking the logarithm of the objective. E-optimal design instead minimizes the norm of E. The resulting optimization problem could be cast as an SDP. A-optimal design minimizes Tr[E], which sets the objective as the mean of the norm of the error squared. Lagrange duals of the three scalarizations have geometric interpretations. D-optimal design, for example determines a minimal volume ellipsoid centered at the origin. The duals of E-optimal and A-optimal designs do similar things (7.5.2).

Each experiment can be associated with a cost c_i representing economic cost or time required w.r.t. v_i. It's essentially a simple weighted sum, m_ic_i = mc^Tλ. If each experiment yields multiple measurements, then this can be generalized to a vector equation with k_i scalar measurements, and this yields a more complicated E with different λ_i, each of which gives the fraction of either the times the first or second experiment should be carried out alone, or together. Which is associated to which is then determined by the practice, not entirely by the maths.