I find two blocks and you reward me with a pool diff of 1280!?

[23:02:38] Pool 0 difficulty changed to 1280
[23:02:38] Accepted 4a93714f Diff 879/128 GPU 1 at Pool 0
[23:02:49] Found block for Pool 0!
[23:02:49] Accepted 3a04fe27 Diff 289K/1280 BLOCK! GPU 0 at Pool 0
[23:03:06] Accepted 03373995 Diff 20.4K/1280 GPU 0 at Pool 0
[23:04:01] Found block for Pool 0!
[23:04:01] Accepted 93e80614 Diff 113K/1280 BLOCK! GPU 1 at Pool 0
[23:05:06] Accepted 0679a5a0 Diff 10.1K/1280 GPU 1 at Pool 0
[23:06:51] Accepted 2ff077e2 Diff 1.37K/1280 GPU 0 at Pool 0
Bah, lightweight:
[18:35:50] Accepted 07d5d5c2 Diff 8.36K/6826 GPU 5 at Pool 0
[18:36:13] Pool 0 difficulty changed to 54608
[18:36:13] Accepted 093976d5 Diff 7.1K/6826 GPU 4 at Pool 0
[18:37:53] Accepted 71a43cff Diff 148K/54608 GPU 4 at Pool 0

Did I miss instructions for individual workers?
No need for separate workers. Well it is still beta but just looking at my CGMiner screen shows me high diff as well for a short time, just before a disconnect.
[00:33:27] Accepted dcb1fece Diff 76K/7880 GPU 1 at Pool 0
[00:43:11] Accepted 0276b16c Diff 26.6K/7880 GPU 1 at Pool 0
[00:45:12] Stratum connection to Pool 0 interrupted
[00:45:14] Pool 0 not responding!
When it came back it was back at 512, then 1706, 989, 2197, 3515 which it is at now.
Reject rate is a perfect 0 after mining for almost 5 hours. Profits however don't look good yet.
Just found another block again as well!

[01:05:09] Accepted 0a45ee76 Diff 6.38K/2197 GPU 1 at Pool 0
[01:05:26] Found block for Pool 0!
[01:05:27] Pool 0 difficulty changed to 3515
[01:05:27] Accepted 15631a82 Diff 784K/2197 BLOCK! GPU 0 at Pool 0
[01:07:04] Accepted 09ba238e Diff 6.74K/3515 GPU 1 at Pool 0