For me "working smart" has two meanings. First one is that "smart" refers to those kind of work, that require you intellect, rather than physical exercises. Intellectual work isn't that difficult, comparing to physical, but it must be appreciated more. And the second meaning, "smart" may also refer to the way you do your work, not the nature of work: you need to find useful contacts, you need to know how to plan your time, you have to be orderly and know how to present yourself.
Personally, I don't think it is the right approach to distinguish between hard work as mostly physical and smart work as mostly intellectual, though it should be clear that when your work involves physical part, for example, as a mover, there is not much you can do with it, I mean, doing it in a smarter way. So we are basically left with intellectual work involving your brain, and that's where most of the distinction should be made between working hard and working smart since you can use your intelligence in a smart or dumb way.