BUT with the Bitcoin blockchain as the secure, censorship-resistant, base layer.
Forking a drivechain into Bitcoin's consensus makes it part of the base layer. That's the issue.
Game-theory. Why would a miner try to "win" the competition by destroying their source of income? Plus, isn't that the situation in most altcoins? What's stopping them?
Altcoins get 51% attacked all the time, and that's just to perform limited double spending attacks.
Why don't we fully trust SPV wallets? Because they trust miners to be honest. The same logic applies in this case, since Bitcoin nodes only SPV validate drivechains.
Such an attack would only destroy faith in drivechains, not Bitcoin, so the "Miners won't 51% attack because of long term incentives" argument doesn't apply. Presumably, transaction activity would not only return to Bitcoin, but at even higher fee rates.
A 51% attack on Bitcoin only allows miners to double spend. On a drivechain, they can just steal everything.
It has been said ever and ever, still a very misleading and wrong assertion.
Paul Sztorc admits it is true
here.
I didn't realize that
he later addressed this by increasing the withdrawal requirement to 13,150 ACKs.
That's one way of sidestepping the whole problem -- nobody will be interested in using this sidechain since it takes 3-6 months to withdraw your bitcoins! Therefore, there will never be enough value to on the drivechain for any of this to matter.

Thus, mis-withdrawals are possible they just take 3-6 months to go through (as do valid-withdrawals).
A hypothetical collusion by miners against a sidechain in the future, will make my above argument void as it proves the mere existence of a 51% that exposes bitcoin to double-spend and censorship threats.
Sidechain theft is much easier than targeted double spend attacks. That's something you aren't accounting for.
Suggesting such hypothetical miner-collusions as "a security hole" or "something concerning" for sidechains is absurd too as it is applicable to every single two-way-pegged solution, the most distinguished one being LN. e.g. they could selectively censor/nullify anti-cheat punishment transactions in favor of their own fraudulent behaviors in the network and our superhero full nodes would have absolutely no clue about the existence of the problem, forget about being helpful.
The rules for sidechain withdrawals aren't enforced by Bitcoin full nodes. That's the difference. Miners can collude to steal all drivechain funds; it's simply a matter of waiting. This situation does not apply to Bitcoin or LN.
What you're suggesting is just attempted censorship and is probably virtually impossible already. It's completely impossible once we're transacting with LN via Taproot.