@Brock_Taylor , @lpb , this seems to be an issue which applies to any number-type custom field, including time tracking.
I see that this has also been reported here, I just tested it!
Indeed strange behaviour when also adding the sorting into the ‘mix’…
I’ve added this to 4.9 in 
List of technical and data limitations