Niceness would be interesting for multi-minute running queries, such as bulk loads or ALTERs; for seconds-scale queries the problem is not so interesting (or interesting in another way).
Therefore one can re-caliber niceness on a second-granularity basis, e.g. every 1 sec. Perhaps for a second or two an operation would be intense, but would afterwards throttle/reduce-priority for the duration of a couple more seconds. Morgan suggests implementation might use tickets-like mechanism; I need to read more about it.

For system that run full power all the time, perhaps the correct solution is to pre-allocate resources; like max IOps.
I personally prefer state-based throttling and not configuration-based throttling, but I guess different workloads would do better with either.

If niceness is set per connection or per query, it might then be possible to ‘CANCEL NICENESS FOR QUERY 12345’ or similar. If niceness is as I suggested above, revisited every 1sec or whatever, then change to niceness would apply in reasonable time and you would feel safer having this escape route.

Using “nice” to slow some queries is an interesting problem. Some queries that run too slow can be less than nice. A long open snapshot blocks purge in InnoDB and causes undo to grow. A long running UPDATE statement might hold row locks longer than desired. Can you be more specific about how you will use this feature?