Looking to move Bitcoin off an exchange

Hi all,

I am in the early stages of Bitcoin with experience only in buying and selling on coinbase. I see all the time “Not Your Keys Not Your Coins” so want to move the little I have onto a Ledger wallet but am curious about why it’s safer.

The way I understand the risk is if Coinbase goes under then so does the Bitcoin I hold with Coinbase. Is this not the same with Ledger? If Ledger goes bust will my hard wallet be useless? Is my Bitcoin only as safe as the company I keep it with?

Thanks for any help.