I have a query to extract data from server. The server contains millions of data and I need to filter out only 56000 of Doc IDs out of those. Could anyone help me build an SSIS? I cannot use Merge here since I would ave to extract the entire data from the server and then merge it with 56000 IDs.
Asked
Active
Viewed 119 times
-4
-
1Can't you use some view? A `WHERE` clause with thousands of parameters looks *very wrong*. – Basile Starynkevitch May 18 '16 at 07:00
-
Could you throw some light one the VIEW part? – Prasanna V May 18 '16 at 07:06
-
GIYF: [SQL: using Views](http://www.tutorialspoint.com/sql/sql-using-views.htm) – Basile Starynkevitch May 18 '16 at 07:34
-
Where do these 50,000 ids come from, the optimal solutions depend on that. For example you may be importing them from excel... – Joe C May 18 '16 at 11:32
1 Answers
1
Put your 56000 IDs into a table called e.g. IDsWanted on the server. Join your data table to IDsWanted in an SQL query, and use this as the source for your SSIS operations.
Obviously a PK on column IDsWanted.ID will help performance of this query.
SebTHU
- 1,385
- 2
- 11
- 22