Sorting large collections with Q

Question

I want to sort large collection of numbers with Q without persisting any intermediate results into tables. How may I sort lists in memory without creating copies of the list at every iteration? Can I update existed li: and modify its elements?

Answer 1

Q does not "persist" intermediate results on a sort... I assume you meant the large memory overhead due to copy?

For asc and desc you have no control over memory, short of re-writing them as c functions in some more memory efficient way.

I guess if you write a sparse-bucket type of sort in c, where for each element you put the numbers into a sparse bucketed list on disk rather than in-memory. Then delete the in-memory list and then read from disk. Sounds very slow though.

I don't see how else you'd do this in q... even iterating through some vector a and iteratively deleting from it whilst creating your sorted list is a problem because you're still making a copy before deletion.