How risky is it really to hodl in a single sig cold wallet?

I am slowly stacking more into a ledger nano x I bought a while ago. it is not too much but it is a significant part of my networth. I have a single sig setup and followed the instructions. I am curious how risky it is to have a single sig cold wallet like ledger? When should one consider more advanced setup like passphrase or multisig? Has there being many cases where someone just got lucky and randomly guessed a private key with coins?

Thanks!