Yep, you hit the wall of the hard fact that at any given moment a file's status could be changed. Some app opened it, some hardware change happened, permissions got changed, etc.
You could test the file for this, or that, and then, when you went to take action, the file has changed state.
There's an interesting article here on CP that tried to address this issue: [
^].
But, even if you use those techniques, I think you still need a Try/Catch. Depending on what the overall goal is here, I'd think about generating summaries of the files that did not get deleted, with relevant error information.