would Linux offer a feature to keep rolling logs that are automatically getting trimmed at the top, e.g. to a fixed file size or to a fixed number of lines?
You can define a log rotation job with logrotate and specify a max file size.
http://linux.die.net/man/8/logrotateLogrotate is a daily cron, though it can be moved to be performed at other frequencies, hourly for example. If I were going to rotate cuda logs I would probably add my own hourly cron rather than modifying logrotate. I think both solutions would present problems if the process maintains an open file descriptor (like it does for stdout, and using > to redirect just redefines stdout) and never gets restarted.
A simple bash script could automatically trim files. Check line count with `wc -l`, if it's above your limit do `tail -n ` and redirect that back to the log, but it will currently do funky things if the file is open/being used by cudaminer. Using that method with a bash redirect will fill the file with 0s (or potentially random data) up to the file seek position maintained by the process. Redirecting the tail into a second file and replacing the first with the second is also no good, you'll stop getting output, but the process still has a fd so the system is still keeping track of a file and potentially writing data to disk, just not to any files you can actually see without mucking through /proc.
Adding an argument for log outputs would go a long way to fix this, but there are a few ways it could actually be handled. Honestly the easiest method (or my impression of the easiest, which is likely naïve as I'm a computer architecture hacker not a sys admin) would be to implement log rotation into cudaminer. Write a wrapper for the logging function, check the seek position (or file size, but that might be more costly), if it's above the limit do a bit of file manipulation (copy the file and reset seek or rename the file and open a new fd).
I'm probably just missing a simpler solution, but I'm a C programmer with a bad habit of reinventing the wheel.