To 'have egg on one's face' means to be embarrased and since those who are embarrased have either been bare-assed in public, lol, or have been caught in a lie or cheating or stealing, etc., then one would do well to exercise a measure of caution around them.