Quote

How can I configure it such that I mine on ASIC difficulty?

Just use "getblocktemplate", and check, which value is displayed there. Now, I don't use "getdifficulty", in the same way, as I don't use Bitcoin Core's fee estimations.

Quote

Is the displayed difficulty determined only by the frequency of the ASIC mined blocks?

Well, the network difficulty was supposed to change once per 2016 blocks (two weeks). Test networks broke that assumption, so if some website assumed that, then it can treat "difficulty=1" incorrectly. The best way is to check "chainwork" instead.

Quote

the difficulty seems way higher than that

If you have more than 80% blocks, mined with the minimal difficulty, then the consequence is that the difficulty, declared in block headers, is measured not as "the difficulty per 10 minutes", but rather "the difficulty per hour" instead. So, if you see "testnet4_diff=6000", then you should read it as something like "testnet4_diff_per_10_minutes=1000" instead.

Think about it in that way: you start from some honest network, where 100% blocks are mined by ASICs. You can see 2016 blocks with real difficulty, per two weeks. If CPU miners can join it, and mine half of the blocks, then you have 1008 CPU blocks and 1008 ASIC blocks per week. Then, the difficulty is doubled. And then, you have 504 ASIC blocks (because of unfairly raised difficulty) and 1008 CPU blocks in the next week, so the next time, the difficulty is raised again. And so on, and so on. Until reaching an equilibrium, which seems to be something around 1/6th of blocks, mined by ASICs. So, instead of having "the difficulty per 10 minutes", you have "the difficulty per hour" instead. And the main rules, which are stopping the difficulty from raising infinitely, are just Median Time Past on the last 11 blocks (strong consensus rule), and 2 hours rule (de-facto standard, non-consensus rule).

Quote

Or are those hashrates displayed not accurate at all?

If you want to get accurate values, then look at chainwork. It can tell you, how many hashes were needed, to make the chain.