Cancelling G Fare - Taxes and Fess Refund?

Is it accurate to say one does not receive a refund for the "taxes and fees" paid to upgrade from PE to UC when cancelling the original flight?

I was booked in PE [S] and used miles to upgrade to UC [G] at a cost of approximately $330 in taxes and fees. Unfortunately I had to cancel the flight but I'm being told the $330 is non-refundable. They were able to give me miles back but is that everyone's experience, they don't refund taxes and fees on a flight you did not take? I received the full amount back for the original booking just not anything paid towards the upgrade. It doesn't seem right but I don't know the rationale behind it. I'm sure this has been asked before but I can't find anything when searching.

https://www.virginatlantic.com/gb/en/fo ... miles.html

4.2. Virgin Atlantic Flight Rewards

4.2.1 Once Reward tickets are issued, you may amend your travel date, route or class of service prior to travel for an amendment fee of £30 for changes to international flights originating in the UK, US$50 for changes to international flights originating in the US or local currency equivalent of US$50 for changes to flights originating in any other region. Where the Reward is a Companion Reward and the person travelling on the paying ticket changes his or her flight or class of travel, the Companion Reward may be similarly amended for the same amendment fees, subject to availability of seats and the provisions of section 4.4 of these Flying Club Terms

Once tickets for a Reward Booking are issued, no name changes are allowed. If for any reason a flight Reward is cancelled by the Member outside 24 hours of departure, 100% of the Miles will be re-credited to the Member’s account. A cancellation fee of £30 for changes to flights originating in the UK, US$50 for changes to flights originating in the US or local currency equivalent of US$50 for changes to flights originating in any other region will apply.