Efficiency of List creation from array

Question

I need to create a list starting from an array already created immediately before and will only be converted to the list. So I could harness the array to the list without making a copy, but the constructor makes a copy. I can even understand the motivation for this. However there are cases that I can guarantee that the array does not have and will have no reference to it other than where it was created from.

Is there any way to make this construction more efficient and use the array internally in the list? I know there are implications if I misuse it.

The most obvious example for this is to get the result of a string.Split(). If you need a list your only obvious way out would be to do this conversion. For now I'm not considering writing a method to split directly into a list.

Try : string.Split().ToList() so you do not create the array. — jdweng, Jan 24 '19 at 18:32
Yes, but it doesn't create a variable. So later in your code you don't have to create the list as a 2nd variable. — jdweng, Jan 24 '19 at 18:52
I think you cannot avoid the copy of an array. Look at https://stackoverflow.com/questions/54298050/fastest-way-from-iterator-to-list-in-c-sharp/54298591#54298591. — Johnny, Jan 24 '19 at 19:06
@Johnny is I know that. But maybe it is a less known alternative. — Maniero, Jan 24 '19 at 19:11
When you add an object to a list is does not make a copy. It uses a link to the original object. — jdweng, Jan 24 '19 at 19:47
@Maniero Generally you could retrieve the internal _items array and use memcpy to copy the array and just update _size field but it is a bit dirty hack...I have managed to do something like that... — Johnny, Jan 24 '19 at 19:59
Most likely this is done for security reasons. Of course, you understand where such code may lead, but some people do not understand this. — Vasek, Jan 24 '19 at 20:05
@jdweng I get an array, I'm not saying about add elements. And it's more complicated that you say. Anyway it's not my problem here. — Maniero, Jan 24 '19 at 20:07
@Johnny I need convert an array to a List not get the underlaying array from List, and I want avoid copy, your solution does copy. — Maniero, Jan 24 '19 at 20:08
@Vasek Not exactally for security but problably it is about protect programmer to shoot yourself having an external reference to underlying array in List. — Maniero, Jan 24 '19 at 20:11
@Johnny My question is more about an official way, but your suggestion could work, I will try. — Maniero, Jan 24 '19 at 20:21
@Maniero I understand, my suggestion is for sure not official, but it works, I will share the solution... — Johnny, Jan 24 '19 at 20:23
If you do not need to change the list later, look at [Array.AsReadOnly](https://learn.microsoft.com/en-us/dotnet/api/system.array.asreadonly?redirectedfrom=MSDN&view=netframework-4.7.2#System_Array_AsReadOnly__1___0___). If I understand everything correctly, there will be no copies — Vasek, Jan 24 '19 at 20:26
Should that "change it later" operation impact the contents of the list as well? Or did you mean "change the LIST later"? In other words, please be specific and please explain *all* your criteria. — Lasse V. Karlsen, Jan 25 '19 at 07:48
Having said that, is there any reason why you did not just create a list to begin with and fill that with data instead of the array? — Lasse V. Karlsen, Jan 25 '19 at 07:48

Johnny · Answer 1 · 2019-01-25T07:22:47.940

As far as I know, there is no official way to do that but it is still possible using System.Reflection. By looking at the source code of the List<T>, .NET Framework 4.7.2, the two important properties are _items and _size. There is also _version but that one changes only when you modify List<T>. Modification are Add, AddRange, Remove, etc. but also Reverse and Sort. So let's assume this is the same operation as creating the list from IEnumerable<T> where _version stays zero.

public static class ListExtensions
{
    public static void SetUnderlyingArray<T>(this List<T> list, T[] array)
    {
        lock (list)
        {
            SetInternalArray(list, array);
            SetInternalArraySize(list, array.Length);
        }
    }

    private static void SetInternalArraySize<T>(this List<T> list, int size)
    {
        var prop = list.GetType().GetField(
            "_size", 
            BindingFlags.NonPublic | BindingFlags.Instance);
        prop.SetValue(list, size);
    }

    private static void SetInternalArray<T>(this List<T> list, T[] array)
    {
        var prop = list.GetType().GetField(
            "_items",
            BindingFlags.NonPublic | BindingFlags.Instance);
        prop.SetValue(list, array);
    }
}

and then set the underlying array

int[] array = Enumerable.Repeat(1, 1000000).ToArray();
List<int> list = new List<int>();

list.SetUnderlyingArray(array);

Note This solution is highly dependent on the details of the implementation and might be wrong if something change in the List<T> internals but it gives insight on how it could be accomplished.

I will try. I don´t want this solution exactly because it is detail dependent. But probably this is the only option. If `_items` were `protected` it would be ease. — Maniero, Jan 24 '19 at 20:42
@Maniero Exactly. I stated that dependency I am aware of that and I do understand your concern... — Johnny, Jan 24 '19 at 20:44

score 1 · Answer 2 · answered Jan 24 '19 at 20:51

1

If you don't specifically need a List<T> You could create a new class that implements IList<T> and doesn't make a copy.

answered Jan 24 '19 at 20:51

nloewen

1,279
11
18

This is an option if everything fails, I thought about that, it-s a radical way, but it's a valid one. – Maniero Jan 24 '19 at 20:54
I'd say it's less radical than a reflection approach. There's no dependency on the implementation of `List` – nloewen Jan 24 '19 at 20:57
Yep, reflection is radical too – Maniero Jan 24 '19 at 22:45

Efficiency of List creation from array

2 Answers2