It’s the little quantity of oil that mars the surface of water.
It is the little fumes from the anus that spoils the buttock. It is the little slack in the foundation that causes a total collapse of the structure. It is not about that little part, it is about the importance of that little part. It is a very wrong perspective to ignore the little part you are engaged in. Though it might seems less relevant, if ignored might ruin the whole. This proverb also emphasises that a little indiscretion damages a good reputation.