We report on an experimental study that explores cross-cultural differences in perception of trustworthiness based on facial traits. In the first part of the experiment, individual male and female neutral photographs of Buryats (Mongolian people of Eastern Siberia) were rated on trustworthiness by men and women from the same population. The trustworthy-looking facial traits were investigated by means of geometric morphometrics, and analysis of the facial action units using artificial neural network (FaceReader). Significant associations between facial traits and perceived trustworthiness were revealed only for male photographs rated by men. Facial shape pattern along trustworthy–untrustworthy vector corresponded to the facial femininity-masculinity vector for Buryats, as well as to the positive-negative vector of the emotional connotation of the neutral facial shape. “Untrustworthy” facial shape was characterized by relatively narrower lower jaw, lower set eyebrows, as well as a lower position of the “Brow Lowerer” facial action unit—a frown. In the second part of the experiment, two geometric morphometric morphs, representing “trustworthy” and “untrustworthy” Buryat male facial shapes, were judged on trustworthiness by male representatives of Buryat, Tuvan (Mongolian people of Southern Siberia), Russian, Indian, and East African (Tanzanians) cultures. The results revealed that in all studied samples the “trustworthy” male portrait was rated significantly higher on trustworthiness than “untrustworthy” one. However, perceived trustworthiness, and agreement of portrait judgments with those of Buryats significantly declined with geographic and genetic distance between populations.