well when do these inactive worker reset / drop forever from the wallet worker history?
Only on pool restart for the most part.
so that means almost never

Please consider having a script purging inactive workers data - might optimise storage but definitely improve user experience. There's definitely a bug with workers stats section - possibly associated with renaming the workers or switching between unnamed and named workers wrt inactive workers.