I beleive the problem is that you are deleting items which then deletes themfrom the list, so you no longer have a valid for each loop.
Move all the items to a new list, and then delete using that list. This can be done easily using the
System.Linq ToList()
method. You can also manually move to a new list.
Dim DEL_LIST = Me.ListView1.SelectedItems.ToList()