It works the same as fields at the project/task level; that is, if you add it via “Portfolios > Workflow > Fields > Add” and you do not check the “Add to [org name]'s field library” checkbox, then it will belong only to that portfolio. If you do check that box, then it’s eligible to be added to other portfolios at some future point.
@ka_nishiyama adding to @Phil_Seeman ’s accurate explanation: The same field can be available for in projects AND portfolios if added to the library, allowing to have tasks converted to projects carry their custom field values with them.
This is great for project pipelines when you go from form to task to project.