I want to store a battery powered device long term (decades) as a reference article, it will never be switched on or charged again. The problem is that it contains a small LiPo battery that will be very hard to remove.
Is there likely to be any significant risk I need to worry about? Once depleted will the battery be relatively inert?
LiPo batteries will not last 10 years, even if unused.
They will self-discharge in a matter of a few years (the exact time this takes depends a lot on the quality of the cell btw.). After you reach a critical level of discharge the electrodes inside the battery will start to become irreversibly damaged (typically below 2.7V but this can vary a little depending on exact type).
after this damage becomes severe enough the cell can be considered dangerous because trying to recharge it at this point could result in an explosion. the same is true if a LiPo cell is damaged by some other method, like crushing or extreme temperatures.
if you do not recharge a cell that was damaged from a ultra deep discharge you are relatively safe though. the most probable danger from such a cell would then be leaking electrolyte because the hermetic seal could be damaged from long-term corrosion / environmental damage.