What is a privileged instruction?

Question

I have added some code which compiles cleanly and have just received this Windows error:

---------------------------
(MonTel Administrator) 2.12.7: MtAdmin.exe - Application Error
---------------------------
The exception Privileged instruction.

 (0xc0000096) occurred in the application at location 0x00486752.

I am about to go on a bug hunt, and I am expecting it to be something silly that I have done which just happens to produce this message. The code compiles cleanly with no errors or warnings. The size of the EXE file has grown to 1,454,132 bytes and includes links to ODCS.lib, but it is otherwise pure C to the Win32 API, with DEBUG on (running on a P4 on Windows 2000).

score 39 · Accepted Answer · edited Jul 14 '13 at 19:36

To answer the question, a privileged instruction is a processor op-code (assembler instruction) which can only be executed in "supervisor" (or Ring-0) mode. These types of instructions tend to be used to access I/O devices and protected data structures from the windows kernel.

Regular programs execute in "user mode" (Ring-3) which disallows direct access to I/O devices, etc...

As others mentioned, the cause is probably a corrupted stack or a messed up function pointer call.

score 7 · Answer 2 · answered Sep 18 '08 at 03:03

This sort of thing usually happens when using function pointers that point to invalid data. It can also happen if you have code that trashes your return stack. It can sometimes be quite tricky to track these sort of bugs down because they usually are hard to reproduce.

score 7 · Answer 3 · answered Sep 18 '08 at 03:04

7

A privileged instruction is an IA-32 instruction that is only allowed to be executed in Ring-0 (i.e. kernel mode). If you're hitting this in userspace, you've either got a really old EXE, or a corrupted binary.

answered Sep 18 '08 at 03:04

Ana Betts

73,868
16
141
209

The exe I compiled using VC6 about 10 seconds ago.... but is does contain links to ODBC.LIB and other libs that might be quite old. This app is not a driver or a service. – David L Morris Sep 18 '08 at 03:10

score 6 · Answer 4 · edited Jun 19 '21 at 10:42

As I suspected, it was something silly that I did. I think I solved this twice as fast because of some of the clues in comments in the messages above. Thanks to those, especially those who pointed to something early in the app overwriting the stack. I actually found several answers here more useful than the post I have marked as answering the question as they clued and queued me as to where to look, though I think it best sums up the answer.

As it turned out, I had just added a button that went over the maximum size of an array holding some toolbar button information (which was on the stack). I had forgotten that

#define MAX_NUM_TOOBAR_BUTTONS  (24)

even existed!

score 4 · Answer 5 · answered Sep 18 '08 at 03:04

First probability that I can think of is, you may be using a local array and it is near the top of the function declaration. Your bounds checking gone insane and overwrite the return address and it points to some instruction that only kernel is allowed to execute.

score 2 · Answer 6 · edited Oct 19 '11 at 19:51

The CPU of most processors manufactured in the last 15 years have some special instructions which are very powerful. These privileged instructions are kept for operating system kernel applications and are not able to be used by user written programs.

This restricts the damage that a user-written program can inflict upon the system and cuts down the number of times that the system actually crashes.

score 2 · Answer 7 · edited Oct 19 '11 at 19:52

2

When executing in kernel mode, the operating system has unrestricted access to both the kernel and the user program's memory.

The load instructions for the base and limit registers are privileged instructions.

edited Oct 19 '11 at 19:52

Peter Mortensen

30,738
21
105
131

answered Dec 12 '10 at 19:18

Bushara

21
1

score 2 · Answer 8 · edited Oct 19 '11 at 19:47

2

I saw this with Visual c++ 6.0 in the year 2000.

The debug C++ library had calls to physical I/O instructions in it, in an exception handler. If I remember correctly, it was dumping status to an I/O port that used to be for DMA base registers, which I assume someone at Microsoft was using for a debugger card.

Look for some error condition that might be latent causing diagnostics code to run.

I was debugging, backtracked and read the dissassembly. It was an exception while processing std::string, maybe indexing off the end.

edited Oct 19 '11 at 19:47

Peter Mortensen

30,738
21
105
131

answered Sep 18 '08 at 03:01

Tim Williscroft

3,705
24
37

It is actually VC6. But not C++, though plenty of zero terminated strings. (I could use the new compilers but have heard rumors that VC6 is actually faster for C, rather than C++). I doubt it is a compiler bug though... (I always discover it is one of those - "what was I thinking" moments). – David L Morris Sep 18 '08 at 03:08

score 2 · Answer 9 · answered Sep 18 '08 at 03:10

2

The error location 0x00486752 seems really small to me, before where executable code usually lives. I agree with Daniel, it looks like a wild pointer to me.

answered Sep 18 '08 at 03:10

Jeremy

2,321
3
21
24

What is a privileged instruction?

9 Answers9

Linked