That's the same with me, yet weirdly I am actually capable of programming >_< I just have to take my time with it and make sure I understand it all.
Programming doesn't really involve much complex math. Basic knowledge can get you very far.
But I do think any intelligent person would have to have a decent comprehension of maths.
You may well be right in that as maths is used in almost everything and is used to explain a lot. Physics is directly linked which in my opinion both are the most interesting and needed subject in the world.