I've done it many time Shawn. It's sort of like the %TEMP% folder in the sense that it should remove OLD entries and only keep new ones. So, I would not recommend deleting the files all the time, but maybe once every month or so and I find that it does improve performance.

The system attempts to manage this for best performance on it's own but sometimes I think it either just gets clogged or lost and doesn't remove ancient entries.

It will take a few reboots and running of applications to reset the performance algorithms to best performance again.