128 bits should be enough, you can put them into an PRNG then. You will not get more security with a secp256k1 ECDSA key anyway.
This is somewhat misleading.
256-bit ECC is normally described to have 2^128 security. But this is in the limit against insanely expensive attacks.
It's also interesting to consider what the chances are of success against an attacker that only does a small amount of work: An attacker who only performs a small amount of computation would have a success rate that reflects a security level similar to 2^256.
As the attacker does more work, their probability of success increases faster than linearly, until success becomes very likely at around 2^128. It's worth mentioning that with only somewhat more than 2^128 work an attacker breaks almost all keys, not just one, so a very high work factor attacker needs basically only operation on average per key they break.
It works like this because the discrete log breaking is effectively a collision search. You build a huge table of points along a path, and when you find that you've looped back to where you've been before, you know that the table passed through the value you started on, and so you can just walk through it to find your discrete log. When your table is small unlikely to find a loop. As it grows sufficiently big almost any point you start with joins into a loop and can be used to break any key fast.
Some attackers are perfectly happy to try negative expectation attacks with a low probability of success just in case-- if you've ever typed a sentence from atlas shrugged into some brainwallet site and checked the address, then maybe you were one of those attackers too, having additional security against them isn't bad, and using full entropy keys can help. That said, I'm sure you could not feel bad at all about using 256 bits of 'untrustworthy randomness' from the computer and then rolling dice for another 128 bits and summing them.
There may also be other attacks on ECDSA if we know that the key is confined to a particular subspace. I'm not aware of any that would be interesting, especially with the data fed into a PRNG, but having a full 256 bits of 'some' kind of entropy probably costs you nothing, and also protects you against things like finding out that popular hex-dice dice were biased and only really had 48 bits of entropy. So why not?