Can the cash value of my whole life insurance policy decrease? If you withdraw money or take loans against your cash value, the cash value of your whole life insurance policy will decrease.