I believe this is a bug that is specific to board views with multiple groups applied, but I am not completely sure that it’s not present elsewhere. I am grouping by a single-select field first, then by assignee (to simulate a classic kanban). When I group only by the field, my columns appear as expected (all options + no value when showing empty groups, only columns with a card present when hiding empty groups).
As soon as I introduce the assignee sub-group, the first group (columns) show all field values, including those that were deleted/deactivated. I confirmed that none of the tasks on the board have any inactive values selected (I could see why you might want to present inactive columns with a card present, when that occurs).
This only seems to happen when the single-select field is the first of the two groups (i.e., the columns).