Yes, inside /mnt/mxe/usr/i686-w64-mingw32.static/include/boost there are many files:
/mnt$ ls /mnt/mxe/usr/i686-w64-mingw32.static/include/
------skipped printout -----
ia64reg.h pshpack4.h xpsrassvc.idl
iaccess.h pshpack8.h ymath.h
iadmext.h pshpck16.h yvals.h
iadmw.h pstore.h zconf.h
iads.h pthread_compat.h zlib.h
icftypes.h pthread.h zmouse.h
honglu@debian:/mnt$
The output that you posted here does not show a "boost" folder in that directory that you did "ls /mnt/mxe/usr/i686-w64-mingw32.static/include/"
You should actually have a directory
/mnt/mxe/usr/i686-w64-mingw32.static/include/boost <<