Thanks for the help indeed. So right now in cgminer I see it set as diff 60, and it is still getting the same average, when it should be at least 100Gh/s. My last question regarding this is how do I set the difficulty exactly? Out of all of my miners, I never had to look into changing that.
You set the minimum difficulty in the My Account page on Slush's web site. Minimum difficulty for each worker will show --- if it hasn't been set specifically; click on Edit for the worker you wish to change, type in the new setting and save. Give it a few hours for things to settle down. You are aiming for Slush to report near enough the same hash rate that CGminer reports, but remember that you are likely to see a lower hash rate anyway at peak internet usage times or if the connection routing is iffy. Try small steps rather than going for broke: in your example I would start with perhaps 75 and see what happens. I found it also works in BTC Guild.