EPPLUS Breaking Charts, formatting, etc. in existing excel file

Question

Using Excel 2013, EPPlus 4.5.3.

I have an existing simple excel file with one datasheet called "Data" that has columns formatted and ready to accept data (columns are set appropriately as Date, Numeric, etc. for the data I will put there).

There are 2 charts in the file on a sheet called "Analysis". The charts reference the rows and columns in the sheet "Data".

In this way, I hope to use EPPLus to just populate rows in the "Data" sheet and the charts will show appropriately. I've tested this manually, entering data and yes, the charts work.

Using this code:

public void CreatePackageWithDataSheet(MemoryStream outputStream, IEnumerable<object[]> data,
        string sheetName = "Data", string firstLine = null, int? firstColumn = null)
    {
        using (var package = new ExcelPackage(outputStream))
        {
            var workSheet = package.Workbook.Worksheets.Add(sheetName);
            workSheet.Cells[(firstLine ?? "A") + (firstColumn ?? 1)] // A1 if not specified
                .LoadFromArrays(data.Skip(1)); // skip types

            package.Save();
        }
    }

Where Ienumerable data is the array of rows and column data I will put in the sheet...

Using this code, the data type formatting of the cells on the sheet "Data" get reset to General, the charts don't show the data, and they lose all their formatting, and even a static image I have in the file gets stretched out of position. The whole thing looks like a mess.

What am I doing wrong? How can I just perform a simple operation of putting data in rows and columns without clobbering the whole rest of the file?