Prior Distributions for Gaussian Models Under Differential Privacy with Bounds on Data Values

Abstract

The Gaussian distribution is often used to model univariate, numerical data, even in contexts where individuals' data values are bounded, e.g., test scores and biological measurements. We consider Bayesian inference for the parameters of Gaussian models when (i) analysts seek a post-processing algorithm that takes noisy sufficient statistics as inputs and produces a posterior distribution of the parameters as output, and (ii) the privacy mechanism enforces bounds on individual data values as a condition for satisfying pure differential privacy. Our primary contributions focus on the specification of prior distributions for model parameters. Specifically, we illustrate that failing to account for the bounds on data values when using informative prior distributions can result in posterior inferences with undesirable properties. Further, we show theoretically that the usual default priors used in the nonprivate setting do not lead to proper posterior distributions, and we suggest a default prior distribution that does. We also show empirically that one can improve the performance of inferential procedures by incorporating the boundedness assumptions into the specification of prior distributions. We present methods for inferring about the mean and variance of a univariate Gaussian model and the coefficients and variance of a linear regression. As we discuss, the general strategies used for the Gaussian models can be adapted for other models when enforcing bounds on the data values to ensure differential privacy.