It seems a bit strange that this isn't something that is built into the UI when you run out of space and want to pick up a new weapon. It is possible to remove weapons and shields from your inventory though.
Start by opening up your inventory screen and navigate to the section you want to remove something from. Interact with the specific item and a new menu will appear next to the item where you can equip or more importantly, drop the weapon. If you select the drop option the item will be removed from Links inventory, freeing it up for you to pick up a new sheld or weapon.