There really is no argument that the best kind of insurance is buying term and investing the difference. You are right -- you are protected for a minimal amount and the extra money earns considerably a lot more. But what comes into play here is plain old human behavior. Would you really be able to invest the difference, regularly, consistently, religiously? We know how "things" always like to come up -- a family member who gets sick, a niece needing tuition, a car that unexpectedly conks out, a kitchen sink that decided to leak and cause water damage to your entire wooden floor. If you know yourself very well and know that you can really abide by the "investing the difference" part, then by all means, go ahead and incur the fees in converting that whole life to term insurance.
Back when I was trying to get insurance for myself (and had very limited money), nothing, not even term insurance, appealed to me. I could pay the premiums, but my income wasn't regular -- so I didn't know if I could keep up with that insurance policy a few years down the road. I searched a bit more and found out that my bank, BPI, had this Save-Up savings account product that had an accompanying Philam Life insurance equivalent to 5x up to 10x your average daily balance. And it covers up to age 60 (there's another one that covers up to age 70, I can't remember it now). No premiums, no medical exams, and life insurance of up to 4 million? Perfect for someone like me. I keep part of my emergency fund in there. If I'll be needing insurance coverage beyond 4 million, well, that's one of the reasons why I invest.
I also just want to share that my parents, in their whole lives, did not get any kind of insurance. It just wasn't in their consciousness. Thankfully, nothing happened to them! They are still able to get insurance in their old age, though, through their investments. Plenty of investment vehicles being offered now have an insurance-linked feature. For example, if you open a time deposit for a 5 million placement, you have life insurance coverage of 1 million while the time deposit is in effect. Or something like that.