So if you build with a different bdb version and then create and uses a wallet with that version, then your wallet cannot be opened with a standard ( bdb v4.8 ) version of bitcoin-core. This could be an issue for you years down the road, when maybe you are trying to restore the wallet on some other computer/os.
Wallets created with different (I used v0.7 and v0.14) versions of the precompiled binaries in Windows can be used with Bitcoin Core made with --with-incompatible-bdb in Ubuntu 14.04, 16.04 and 18.04. At least I didn't encounter any obvious problem.
But it's potential incompatibility, risk that some wallet functions may not work. And risk that a wallet can't be opened with future versions.