Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.
  1. What is the difference between DEP always on ("/NoExecute=AlwaysOn" in boot.ini) and DEP opt-out ( "/NoExecute=OptOut" in boot.ini) with no exceptions?

    "no exceptions" = empty list of programs for which DEP does not apply. DEP = Data Execution Prevention (hardware).

    One would expect it to work the same way, but it makes a difference for some applications:

    E.g. for all versions of UltraEdit 14 (14.2). It crashes at startup for DEP always on, at least on Microsoft Windows XP Professional Edition x64 edition. (Update 2010-03-11: this problem has been fixed with UltraEdit 15.2 and later.)

    Update 2010-07-12: Mozilla Thunderbird 2.0.0.19 also crashes at startup for DEP set to the highest level (as UltraEdit used to do).

  2. Is there a difference between Windows XP, Windows Vista and Windows 7?

  3. Is there a difference between 32 bit and 64 bit versions of Windows?

share|improve this question
add comment

6 Answers

Configuration Description

OptIn This setting is the default configuration. On systems with processors that can implement hardware-enforced DEP, DEP is enabled by default for limited system binaries and programs that "opt-in." With this option, only Windows system binaries are covered by DEP by default.

OptOut DEP is enabled by default for all processes. You can manually create a list of specific programs that do not have DEP applied by using the System dialog box in Control Panel. Information technology (IT) professionals can use the Application Compatibility Toolkit to "opt-out" one or more programs from DEP protection. System compatibility fixes, or shims, for DEP do take effect.

AlwaysOn This setting provides full DEP coverage for the whole system. All processes always run with DEP applied. The exceptions list to exempt specific programs from DEP protection is not available. System compatibility fixes for DEP do not take effect. Programs that have been opted-out by using the Application Compatibility Toolkit run with DEP applied.

AlwaysOff This setting does not provide any DEP coverage for any part of the system, regardless of hardware DEP support. The processor does not run in PAE mode unless the /PAE option is present in the Boot.ini file.

share|improve this answer
add comment
up vote 1 down vote accepted

For question 1: I think this difference is caused by the backdoors that Microsoft has put into hardware DEP for OptOut, according to Fabrice Roux (see below). In the case of IrfanView, for which Steve Gibson observed the same difference as I did for UltraEdit (see the question), the difference is caused by a non-DEP aware EXE packer (ASPack) that Microsoft coded a backdoor for.

Sources:

From http://blog.fabriceroux.com/index.php/2007/02/26/hardware_dep_has_a_backdoor?blog=1, "Hardware DEP has a backdoor" by Fabrice Roux. 2007-02-26. "IrfanView was not using any trick to evade DEP ... Microsoft just coded a backdoor used only in OPTOUT. Bascially Microsoft checks the executable header for a section matching one of the 3 strings. If one these strings is found, DEP will be turned OFF for this application by windows. ... 'aspack', 'pcle', 'sforce'"

From http://www.grc.com/sn/sn-078.htm, by Steve Gibson. "I can’t find any documentation on Microsoft’s site anywhere, because we’re seeing a difference between always-on and opt-out. That is, you would imagine that always-on mode would be the same as opting out if you weren’t having any opt-out programs. It turns out it’s not the case. For example ... the IrfanView file viewer ... runs fine in opt-out mode, even if it has not been opted out. But it won’t launch, Windows blocks it from launching ... in always-on mode."

From http://www.grc.com/sn/sn-083.htm, by Steve Gibson. "... IrfanView ... won’t run with DEP turned on. It’s because it uses an EXE packer, an executable compression program called ASPack. And it makes sense that it wouldn’t because naturally an executable compressor has got to decompress the executable, so it allocates a bunch of data memory into which it decompresses the compressed executable, and then it runs it. Well, it’s running a data allocation, which is exactly what DEP is designed to stop. On the other hand, UPX, which is actually the leading and most popular EXE compressor, it’s DEP- compatible because those guys realized, hey, when we allocate this memory, we should mark the pages as executable."

share|improve this answer
add comment

Setting the System DEP Policy level to OptOut allows for any 32bit thread to call SetSystemDEPPolicy(0) which will disable DEP protection for the memory space allocated to that process. If the system's DEP Policy is set to AlwaysOn, invocations of SetSystemDEPPolicy will throw an error.

share|improve this answer
add comment

A detailed description of the Data Execution Prevention (DEP) feature in Windows XP Service Pack 2, Windows XP Tablet PC Edition 2005, and Windows Server 2003

The shorter, snappier title is MS KB 875352 ;-)

More than your ever needed to know...

share|improve this answer
    
I knew about this document. It does not answer question 1 (except perhaps for "AlwaysOn ... System compatibility fixes for DEP do not take effect" - I am not sure how to interpret that). Regards, Peter. –  Peter Mortensen May 29 '09 at 19:32
add comment

There's a fairly big discussion about this on Steve Gibson's securable homepage. Also, they've discussed it quite often in the security now podcast. You can find the transcripts also on grc.com

share|improve this answer
    
I have listened to all the "Security Now!" podcasts regarding DEP. It does not, I far as I know, answer question 1: why is there a difference between AlwaysOn and OptOut with no exceptions. –  Peter Mortensen May 29 '09 at 19:37
    
That's unfortunate... I didn't recall it in that detail. Just one more chance: You also mean it has not been discussed in the listener q&a episodes coming after the "official" DEP episodes, right? (the transcripts help to browse quickly through them) –  Olaf May 30 '09 at 7:17
    
OK, Steve Gibson actually observed the same difference as I did (episode 78 and episode 83). I am not sure if he found the reason or not. But I think Fabrice Roux did! - see beginning of this page. Regards, Peter. –  Peter Mortensen May 30 '09 at 21:20
add comment

KB 875352 is actually the one: OptOut lets you specify a list of programs for which DEP does not apply, AlwaysOn will apply DEP to everything always.

share|improve this answer
1  
But then OptOut with no* exceptions (empty list of programs for which DEP does not apply) should be equivalent to AlwaysOn. This has been observed to not be the case on Windows XP 64 bit. UltraEdit 14 crashes on startup for AlwaysOn, but not for OptOut with no exceptions. I agree: the official documentation suggest the two cases to be equivalent. But they are not. The only change is to the boot.ini line and the problem is reproduciple. I would like to know the underlying reason for this behavior. Regards, Peter. –  Peter Mortensen May 29 '09 at 20:09
    
That's interesting... I know that there is a DEP-aware option in Visual Studio, but what it does seems poorly documented. Perhaps if the app was compiled with this option switched on or off it may also be a contributing factor (in which case there's not too much you can really do about it). –  Jimmy Shelter May 29 '09 at 21:24
    
The DEP-aware option in Visual Studio merely shows that the developer claims that his code works with DEP turned on. It's explicit opt-in for that app. –  Roger Lipscombe Jun 1 '09 at 15:03
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.