TY - BOOK
T1 - Asymptotic Behavior of Some Bayesian Nonparametric and Semi-parametric Procedures
AU - Wu, Yuefeng
N1 - This dissertation extends some established results about the asymptotic behavior of Some Bayesian Nonparametric and Semi-parametric Procedures in three aspects. First, positivity of the prior probability of Kullback-Leibler neighborhood around the true density, commonly known as the Kullback-Leibler property, plays a fundamental role in posterior consistency.
PY - 2009
Y1 - 2009
N2 - This dissertation extends some established results about the asymptotic behavior of Some Bayesian Nonparametric and Semi-parametric Procedures in three aspects. First, positivity of the prior probability of Kullback-Leibler neighborhood around the true density, commonly known as the Kullback-Leibler property, plays a fundamental role in posterior consistency. A popular prior for Bayesian estimation is given by a Dirichlet mixture, where the kernels are chosen depending on the sample space and the class of densities to be estimated. The Kullback-Leibler property of the Dirichlet mixture prior has been shown for some special kernels like the normal density or Bernstein polynomial, under appropriate conditions. We obtain easily verifiable sufficient conditions, under which a prior obtained by mixing a general kernel possesses the Kullback-Leibler property. We study a wide variety of kernels used in practice, including the normal, $t$, histogram, gamma, Weibull densities and so on, and show that the Kullback-Leibler property holds if some easily verifiable conditions are satisfied at the true density. This gives a catalog of conditions required for the Kullback-Leibler property, which can be readily used in applications. Second, the Bayesian approach to analyzing semi-parametric models are gaining popularity in practice. For the Cox proportional hazard model, it has been shown recently that the posterior is consistent and leads to asymptotically accurate confidence intervals under a Levy process prior on the cumulative hazard rate. The explicit expression of the posterior distribution together with independent increment structure of Levy process play a key role in the development. However, except for one-dimensional linear regression with an unknown error distribution and binary response regression with unknown link function, even consistency of Bayesian procedures has not been studied for a general prior distribution. We consider consistency of Bayesian inference for several semi-parametric models including multiple linear regression with an unknown error distribution, exponential frailty model, generalized linear model with unknown link function, Cox proportional hazard model where the baseline hazard function is unknown, accelerated failure time models and partial linear regression model. We give sufficient conditions under which the posterior distribution of the parametric part is consistent in the Euclidean distance while the non-parametric part is consistent with respect to some topology such as the weak topology. Our results are obtained by verifying the conditions of an appropriate modification of a celebrated result of Schwartz. Our general consistency result applies also to the case of independent, non-identically distributed observations. Application of our theorem requires showing the existence of exponentially consistent tests for the complement of the neighborhoods of the "true" value of the parameter and the prior positivity of a Kullback-Leibler type of neighborhood of the true distribution of the observations. We construct the required tests and give sufficient conditions for positivity of prior probabilities of Kullback-Leibler neighborhoods in all the examples we consider in the corresponding chapter of this dissertation. Third, Dirichlet mixtures has been used for multivariate density estimation in practice for quite some time. However, the consistency of such model has not been studied. Valuable results have been given on posterior consistency of Dirichlet mixtures in univariate density estimation. But these results cannot be generalized directly to multivariate cases. By controlling the tail behavior of the base measure of the Dirichlet process, and through the technique of calculating entropy, we give sufficient conditions on the true density and the model prior, under which the posterior consistency holds.
AB - This dissertation extends some established results about the asymptotic behavior of Some Bayesian Nonparametric and Semi-parametric Procedures in three aspects. First, positivity of the prior probability of Kullback-Leibler neighborhood around the true density, commonly known as the Kullback-Leibler property, plays a fundamental role in posterior consistency. A popular prior for Bayesian estimation is given by a Dirichlet mixture, where the kernels are chosen depending on the sample space and the class of densities to be estimated. The Kullback-Leibler property of the Dirichlet mixture prior has been shown for some special kernels like the normal density or Bernstein polynomial, under appropriate conditions. We obtain easily verifiable sufficient conditions, under which a prior obtained by mixing a general kernel possesses the Kullback-Leibler property. We study a wide variety of kernels used in practice, including the normal, $t$, histogram, gamma, Weibull densities and so on, and show that the Kullback-Leibler property holds if some easily verifiable conditions are satisfied at the true density. This gives a catalog of conditions required for the Kullback-Leibler property, which can be readily used in applications. Second, the Bayesian approach to analyzing semi-parametric models are gaining popularity in practice. For the Cox proportional hazard model, it has been shown recently that the posterior is consistent and leads to asymptotically accurate confidence intervals under a Levy process prior on the cumulative hazard rate. The explicit expression of the posterior distribution together with independent increment structure of Levy process play a key role in the development. However, except for one-dimensional linear regression with an unknown error distribution and binary response regression with unknown link function, even consistency of Bayesian procedures has not been studied for a general prior distribution. We consider consistency of Bayesian inference for several semi-parametric models including multiple linear regression with an unknown error distribution, exponential frailty model, generalized linear model with unknown link function, Cox proportional hazard model where the baseline hazard function is unknown, accelerated failure time models and partial linear regression model. We give sufficient conditions under which the posterior distribution of the parametric part is consistent in the Euclidean distance while the non-parametric part is consistent with respect to some topology such as the weak topology. Our results are obtained by verifying the conditions of an appropriate modification of a celebrated result of Schwartz. Our general consistency result applies also to the case of independent, non-identically distributed observations. Application of our theorem requires showing the existence of exponentially consistent tests for the complement of the neighborhoods of the "true" value of the parameter and the prior positivity of a Kullback-Leibler type of neighborhood of the true distribution of the observations. We construct the required tests and give sufficient conditions for positivity of prior probabilities of Kullback-Leibler neighborhoods in all the examples we consider in the corresponding chapter of this dissertation. Third, Dirichlet mixtures has been used for multivariate density estimation in practice for quite some time. However, the consistency of such model has not been studied. Valuable results have been given on posterior consistency of Dirichlet mixtures in univariate density estimation. But these results cannot be generalized directly to multivariate cases. By controlling the tail behavior of the base measure of the Dirichlet process, and through the technique of calculating entropy, we give sufficient conditions on the true density and the model prior, under which the posterior consistency holds.
UR - https://repository.lib.ncsu.edu/handle/1840.16/4681
M3 - Doctoral Thesis
ER -