The sensitive information leakage and security risk is a problem from which both individual and enterprise suffer in massive data collection and the information retrieval by the distrusted parties. In this paper, we focus on the privacy issue of data clustering and point out some security risks in the existing data mining algorithms. Associated with cryptographic techniques, we initiate an application of random data perturbation (RDP) which has been widely used for preserving the privacy of individual records in statistical database for the distributed data clustering scheme. Our scheme applies linear transformation of Gaussian distribution perturbed data and general additional data perturbation (GADP) schemes to preserve the privacy for distributed kernel density estimation with the help of any trusted third party. We also show that our scheme is more secure against the random matrix-based filtering attack which is based on analysis of the distribution of the eigenvalues by using two RDP methods.