In this thesis I develop novel Bayesian inference approaches for some typical data analysis problems as they arise with biomedical data. The common theme is the use of flexible and semi-parametric Bayesian models and computation intensive simulation-based implementations. In chapter 2, I propose a new approach for inference with multivariate ordinal data. The application concerns the assessment of toxicities in a phase III clinical trial. The method generalizes the ordinal probit model. It is based on flexible mixture models. In chapter 3, I develop a semi-parametric Bayesian approach for bio-panning phage display experiments. The nature of the model is a mixed effects model for repeated count measurements of peptides. I develop a non-parametric Bayesian random effects distribution and show how it can be used for the desired inference about organ-specific binding. In chapter 4, I introduce a variation of the product partition model with a non-exchangeable prior structure. The model is applied to estimate the success rates in a phase II clinical of patients with sarcoma. Each patient presents one subtype of the disease and subtypes are grouped by good, intermediate and poor prognosis. The prior model respects the varying prognosis across disease subtypes. Two subtypes with equal prognoses are more likely a priori to have similar success rates than two subtypes with different prognoses.