-10

I have worked with FoxPro databases, which uses the Rushmore optimization technology and I wanted to know if there is any optimization technology for LINQ.

I am not looking for this in LINQ-to-SQL, because Rushmore was actually assimilated into SQL Server, and is responsible for part of its index-related speed.

I want to know for LINQ-to-Objects, if there is something similar to Rushmore or the index-related performance optimizations in SQL Server?

This question is not really a duplicate because 1.) Rushmore automatically optimized your expressions (and IF you do that with I4O, it is done manually), because 2). there was a bitmapped component that allowed multiple indexes to be quickly combined in expressions (and had good performance), and because 3). the technology works for tables that can't fit in memory (which would be a plus, in this case).

MicroservicesOnDDD
  • 332
  • 2
  • 12
  • 4
    This feels like a XY Problem - https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem . Do you have some code that is slow that you are trying to speed up? Why are you asking the question? – mjwills Jul 15 '19 at 23:38
  • Answering your question directly - I am not aware of such a thing, no. – mjwills Jul 15 '19 at 23:38
  • 1
    Do you know how database indexes work? If you did, I'm guessing that you'd realise why that question doesn't really make sense. If you're using LINQ to Objects for the sort of things that indexes would provide a significant advantage then you're probably doing it wrong to begin with. – jmcilhinney Jul 16 '19 at 00:12
  • I would think that something like this would have to be based on `Dictionaries`, where, for example, a `List` was accompanied by a `Dictionary` for each "indexed" property on type `T`. The overhead of creating and maintaining the `Dictionaries` would be significant though, and would only improve performance significantly if you were dealing with a very large collection or a very complex query. In the first case at least, you probably shouldn't be using L2O in the first place. – jmcilhinney Jul 16 '19 at 02:22
  • 1
    This question is being discussed on [meta](https://meta.stackoverflow.com/questions/388655/what-does-too-broad-mean-in-this-context-and-why-was-my-question-put-on-hold-fo#388655) – weegee Aug 19 '19 at 17:05

2 Answers2

2

There is no Query Optimizer and no Indexes in Linq-to-Objects. You can use the ToDictionary, ToLookup, ToHashset extension methods to create "indexes" over in-memory collections, and you can create sorted collections of objects.

You can then manually write queries and procedural code using these optimized collections to replicate what a query optimizer would otherwise do.

David Browne - Microsoft
  • 80,331
  • 6
  • 39
  • 67
  • How would we then efficiently combine index-optimizable expressions as Rushmore seemed to do so well? Object-by-object would be too slow, and perhaps take too much memory. Very nice to get an answer directly from Microsoft, by the way! (Thank you!) – MicroservicesOnDDD Jul 17 '19 at 05:51
-2

(I am just thinking out loud and would be a mess as a comment)

Rushmore optimization was basically about choosing the correct indexes, and\or use no index at all and do a bit map indexing on the fly. While it is a nice technique, I think databases got to speed via their own different tricks besides the indexes themselves. For example, lateral in ansi sql, range index in postgreSQL, shards are few to name. If you use Linq against a particular backend, you would be utilizing that backend's capabilities (including Rushmore, if you are using Linq To VFP).

For Linq To Objects, there isn't such a thing AFAIK, but as a developer you could take the responsibility of writing it as optimized as possible and if you think, L2O is an in memory thing, you might not need it as much as you do with a database. Even with Rushmore, we were taking the responsibility of trying alternative ways of querying for the best performance.

(You have the question tagged as "Linq", I would hope Joseph Albahari -author of .Net xx in a nutshell- would see it and provide an answer in detail)

Cetin Basoz
  • 22,495
  • 3
  • 31
  • 39
  • I like the "LINQ to VFP" possibilities -- do you know of this being done? – MicroservicesOnDDD Aug 19 '19 at 13:54
  • 1
    @MicroservicesOnDDD, what do you mean? Are you asking if "Linq To VFP" and\or Linq To EF for VFP exists? If so, yes, check out Tom Brother's GitHub page (https://github.com/tombrothers). – Cetin Basoz Aug 19 '19 at 16:28