Azure Data Factory - Self-Hosted Integration Runtime - ODBC driver mystery

Question

We are using a Self-Hosted Integration Runtime for Azure Data Factory.

On that machine there was installed an Exasol ODBC driver of version 6. We wanted to upgrade the driver, deleted an old one and installed a new driver of version 7.

Weird thing is that now in Exasol logs we can see that Data Factory is sometimes connecting via driver version 7, and sometimes via driver version 6.

I made an experiment and deleted Exasol ODBC driver from the machine completely. After that Data Factory still was able to connect to Exasol using the driver I just deleted.

Looks like drivers' DLLs are cached somewhere. What can it be?

Update 1

I captured following actions in Process Monitor when Data Fatory connected to Exasol with ODBC driver of version 6:

Where these C:\Config.Msi\3739be5*.rbfASolution-6.1\ODBC\ DLLs may come from? There is no C:\Config.Msi\ directory on the machine.

Update 2

I noticed that when I test connection via Microsoft Integration Runtime Configuration Manager on the machine or in Data Factory Linked Service, then connection is always performed with ODBC driver of version 7.

But when I test connection via Data Factory Dataset, then in some cases connection is done with ODBC driver of version 6.

score 0 · Answer 1 · answered Jun 08 '22 at 08:24

0

You could check the registry but clean at your own risk. An alternative might be the SysIternals tools, Process Monitor or Process Explorer which might help you get to the bottom of this. Install them on the SHIR VM if you are allowed to. Process Explorer in particular is a bit like SQL Profiler (if you've ever used that) so will be able to tell you which registry keys external processes are using. It will give you a lot a lot of information so you will have to make judicious use of timestamp and filtering. The proposed steps:

Start a trace using Process Monitor
Start a pipeline using the Exasol driver
Wait til it completes (or at least you know it has started)
Stop the Process Monitor trace Spend time going through the millions of records it has captured, trying to filter down, or search for your process

An alternative would be to build a clean SHIR and install only the new driver. Then swap it in for the old one. You may have to get the new SHIR added to the firewall if this is an issue for you.

Honestly I would propose both of these approached in parallel for a production problem. Procmon / Process Explorer can be quite labour and time expensive but should help you get to the bottom of the issue. Building a cleaner SHIR is probably a safer option in the long-term, but requires new infrastructure.

answered Jun 08 '22 at 08:24

wBob

13,710
3
20
37

I captured some events. Please, see updated question. – GriGrim Jun 08 '22 at 09:09
Maybe it’s there but you can’t see it? What’s in the .ini file? – wBob Jun 08 '22 at 10:47
Which .ini file? I see hidden items, but there is no .ini file in C:\ – GriGrim Jun 08 '22 at 12:58
The first record in your screenprint lists an .ini file `ExaSolution driver.ini`. I was wondering what was in it. – wBob Jun 08 '22 at 14:27
1

There is no such file. You can see in the next column `NAME NOT FOUND`. As I understand, Integration Runtime first tries to use drivers it knows about, and if nothing is found then looks in the registry. – GriGrim Jun 08 '22 at 14:47
How’s the build of the new one going? – wBob Jun 09 '22 at 06:43
I'm not the one who configured Integration Runtime. So, I'd like not to touch it. – GriGrim Jun 09 '22 at 08:06
See another observation in 2nd update of the question. – GriGrim Jun 09 '22 at 09:20
Interesting. It sounds safer to have a 'clean' SHIR stood up with only version 7 of the driver on, don't you think? – wBob Jun 09 '22 at 12:20
Restarting the SHIR server helped – GriGrim Jun 15 '22 at 08:35

score 0 · Accepted Answer · answered Jun 15 '22 at 08:33

It may sound silly, but rebooting the server where SHIR is working solved the problem. We noticed, that this server was running for more than 30 days, and decided to reboot it. Maybe restarting Integration Runtime service itself would also help, but we didn't do it. Thanks to everyone for you help.

Azure Data Factory - Self-Hosted Integration Runtime - ODBC driver mystery

2 Answers2