ok, several things:

first, the fact is the current mechanism's goal is that we don't need to do any communication between sender and receiver since otherwise we might as well communicate a new address.

so, having to scan all tx and having no way to pre-filter makes it unfeasible from the (light) client side, I have to scan all incoming tx and will have to calculate on all of them to make sure there is no stealth. with the other method, we just need to scan all tx but calculate just on OP_RETURN against the next output. also a matrix op_return, address can be generated to avoid needing to download that much data to scan.

so, having a stealth matrix in the server is a not requisite with the current darkwallet method (just helps for light clients), if you're willing to scan all tx then no need for server table, it just makes it more convenient. and also, even if you need to scan all tx you can still discard most as not having stealth, if they do we still dont need to calculate all in*out to make sure.

also little bug (i think), the mechanism how you avoid address reuse:

vout_priv = sha256((scan_pubkey + prev_hash + vout + index).decode('hex'))

maybe you want scan_privkey instead of pubkey there otherwise the secret is public...