The “Number” field type allows you to set a format of the field, such as number, percent, or various currencies. Because the “Formula” field type primarily only supports numbers, the “Number” formatting should also be able to apply to the “Formula” fields.
It generally does a good job of trying to figure out what format you want based on the fields you use in the formula, but if you want to do things like a “% over/under Budget” field by dividing the “Cost” field by the “Budget” field, it displays as a decimal and cannot be formatted to a percentage. There are other scenarios where it tries to force a specific format based on what it thinks you want, but isn’t the format you need.