Bug 33659

Informational

GemStone/S

6.1.5, 6.1.4, 6.1.3

Windows

GemStone/S with DEP on Windows 2003

Windows 2003 SP1 by default has enabled a security feature, DEP, that prevents code execution from data pages.  GemStone’s native code generation facility violates this security, and as a result GemStone processes may be terminated on Windows 2003 SP1.

Workaround

There are several ways to set up DEP and GemStone that will allow GemStone to run successfully.

1. DEP turned on for essential Windows programs and services only.

This turns off DEP for applications such as GemStone.  While this is the simplest way to permit GemStone to run, it means you are not able to take advantage of the improved security that DEP provides.

To set up DEP for this option:
•     Go to System Properties -> Advanced Tab -> Performance Settings -> Data Execution Prevention Tab.
•     Select the radio button “Turn on DEP for essential Windows programs and services only”
•     You may need to reboot for this to take effect.

2. Turn DEP on, and specifically exclude GemStone executables

In this configuration, DEP is turned on, but the GemStone executables gem.exe, topaz.exe, and (if you are running GemBuilder for Java) gbjd.exe and stopgbj.exe, are excluded.  If you are using GCI application that will load a linked GCI, you need to add this to the exclusion list also.

To set up DEP for this option:
•     Go to System Properties -> Advanced Tab -> Performance Settings -> Data Execution Prevention Tab.
•     Select the radio button “Turn on DEP for all programs and services except those I select:”
•     Use “Add” to trace the file path to the location of the .exe file to be excluded.
•     You may need to reboot for this to take effect.

2. Turn DEP on, do not exclude GemStone executables, and run with no native code generation.

Disabling native code generation allows GemStone to run under DEP without further configuration.

To turn off native code generation, set the following configuration parameters:
GEM_NATIVE_CODE_THRESHOLD = -1
GEM_NATIVE_CODE_MAX = 0

These gem configuration parameters must be set in the configuration file that is used by gems.  One simple way to do this is to set the environment variable GEMSTONE_EXE_CONF to the path and filename of the new configuration file.  See the System Administration Guide, Appendix A, for other options.

Note that stone shutdown requires a gem to be spawned.  If you do not globally specify the conf file location and you encounter an unexpected error on shutdown, the way you are specifying the configuration file may not be adequate for this special case.

To set up DEP for this option:
•     Go to System Properties -> Advanced Tab -> Performance Settings -> Data Execution Prevention Tab.
•     Select the radio button “Turn on DEP for all programs and services except those I select:”
•     De-select all GemStone processes.
•     You may need to reboot for this to take effect.


Last updated: 8/17/07