Delivery and body communication can affect your speech in many ways. It can also contradict your speech in many ways. If your tone is very mellow and lazy, but the topic is very light-hearted and exciting, your delivery will contradict your speech. If your body language is very shy and timid but you have a powerful and important topic, your body language would contradict your speech. When the audience sees these contradictions, it can be confusing as to what the topic is really about and the importance of it.
I agree that the body language and tone contradicting the purpose of your speech can be difficult for the audience to receive. Match the tone and type of speech you are delivering is very important.
It is so true that a contradicting body and speech can be disorienting and distracting from the main point of the speech.