Deleting a file with File.Delete throws System.IO.IOException

Question

I have a user control created by me, which contains Image, TextBlock and TextBox. The idea was to write an application which could read all image files from a directory and for each one creates user control with path to the file as parameter.

namespace ImageViewerMother
{
    /// <summary>
    /// Interaction logic for ImageRep.xaml
    /// </summary>
    public partial class ImageRep : UserControl
    {
        public string path { get; set; }
        public string name { get; set; }

        public ImageRep(string path)
        {
            this.path = path;            

            InitializeComponent();
            InitializeImage(path);
            SettingName();
        }

        private void InitializeImage(string path)
        {
            ImageHolder.Source = new BitmapImage(new Uri(path, UriKind.Absolute));
            RenderOptions.SetBitmapScalingMode(ImageHolder, BitmapScalingMode.Fant);
            ImageHolder.Height = 150;
            ImageHolder.Width = 150;
        }

        private void SettingName()
        {
            this.name = io.Path.GetFileNameWithoutExtension(path);
            NameHolder.Text = name;
            NameHolderChange.Text = name;
        }

        private void NameHolder_PreviewMouseDown(object sender, MouseButtonEventArgs e)
        {
            NameHolder.Visibility = Visibility.Collapsed;
            NameHolderChange.Visibility = Visibility.Visible;
        }

        private void NameHolderChange_KeyDown(object sender, KeyEventArgs e)
        {
            if (e.Key == Key.Return)
            {
                //CREATING NEW PATH TO RENAMED FILE
                string newPath = io.Path.GetDirectoryName(path).ToString() 
                    + @"\" + NameHolderChange.Text + io.Path.GetExtension(path);
                //CREATING COPY OF THE FILE WITH NEW NAME
                io.File.Copy(path, newPath);
                //REDIRECTING IMAGE TO A NEW SOURCE
                InitializeImage(newPath);
                //DELETING OLD FILE
                //GC.Collect();
                //GC.WaitForPendingFinalizers();
                io.File.Delete(path);
                //REPLACING OLD PATH WITH THE NEW ONE AND 
                //ACTUALIZING NAME OF THE FILE IN TEXTBOX AND BLOCK
                path = newPath;
                SettingName();
                NameHolder.Visibility = Visibility.Visible;
                NameHolderChange.Visibility = Visibility.Collapsed;
            }
        }
    }
}

So, when user clicks on TextBlock, it collapses, and TextBox becomes visible, allowing to write a new name for the file. When user presses enter, the application generates new copy of the file with new name, image source redirects to the new source, and old file gets deleted. But I can't delete the file because I get an unhandled exception:

A first chance exception of type 'System.IO.IOException' occurred in mscorlib.dll
An unhandled exception of type 'System.IO.IOException' occurred in mscorlib.dll

Additional information: The process cannot access the file 'E:\foldery\123fgdg\43_460s.jpg' because it is being used by another process.

I read about it on stackoverflow and there are many answers, but none of them are like mine. I don't have an instance of any stream accessing the file, Image component doesn't use it's path as source anymore. I'm pretty sure my program blocks access to the image, because when error occures and I try to delete the file from window explorer, I can't access it, but when I close my app I can delete it without problem.

And one more piece of code - here's how I am getting paths to files:

string[] unfilteredFiles = Directory.GetFiles(newLocationFolder);
foreach (string file in unfilteredFiles)
{
    ImageRep imageRep = new ImageRep(file);
    IMGWrapHolder.Children.Add(imageRep);
}