The difficulty will reset to 1 if the time since the last block is more than 20 minutes. There is no stipulation that after a difficulty reset block that the next block must be the normal difficulty; if the next block is more than 20 minutes after the current block, then it can also have a difficulty of 1.
For blocks that are found within 20 minutes of each other, the block's difficulty will be the same as the difficulty of the last block in the difficulty interval whose difficulty was not 1 OR the difficulty of the first block in the difficulty interval. This behavior is defined here:
https://github.com/bitcoin/bitcoin/blob/master/src/pow.cpp#L32. Because the difficulty look back is clamped to be the first block in the interval, if that block was found more than 20 minutes after the block before it, its difficulty will be 1, and so will all subsequent blocks in the difficulty period. After 2016 blocks, the difficulty adjustment will kick in and bring the difficulty back up.
So the behavior we are observing is that the first block of a difficulty interval (block height % 2016 == 0) is found 20 minutes or more after the previous block, so its difficulty is 1. Then blocks that are mined after it use the difficulty of 1 because that is the difficulty of the first block in the interval.
As far as I know, the difficulty of a block multiple of 2016 doesn't change if it takes more than 20 minutes to process. Such blocks are the only ones not affected by that rule in the testnet.
The difficulty of such blocks is calculated depending on the difficulty of the block immediately prior to them. If that block took more than 20 minutes, then this block
could have the difficulty set to 1 as well, and so would the blocks mined after this one.
I said "could" because, if the previous 2,016 blocks took less than 2 weeks to process, then the difficulty could be slightly greater than 1.