The simplest and most direct way to permanently delete files or folders is by using the Shift + Delete keyboard shortcut.

You can give your system a "mini-reset" with Task Scheduler every time you log in. This can help you clear temporary files, ...