The site uses shares submitted, which isn't really a reflection of your "hashrate" reported by your miner - it is what your miner calls work units. On my own Gridseeds, I've noticed that if I use higher difficulty I see a big variance in my shares submitted per 10 minutes. I use 128 for my 6MH and 10MH rigs of Gridseeds. For less than 16 of the 5-chip Gridseeds, I would use 64 for your difficulty.
I have 20 GS grids each as one worker with your pool. Is there a place I can see shares submitted then instead of hash rate if it's not the most useful of number? If I increase my difficulty you're saying I'll see a hashrate closer reflecting what I see locally, but it sounds like that doesn't really matter, more of an appearance issue?